Could technology help reconnect indigenous communities with a rich – but vanishing

A new approach to anthropology that seeks to build connected virtual worlds may help one of South America’s Amazonian communities preserve their heritage.

What’s more, the creators of these virtual worlds believe the tech they have used to create them – a combination of graph database, Generative AI, and Virtual Reality (VR) tooling – could also be connected to gaming engines to create truly immersive worlds that outsiders could ‘visit’ to learn from.

As one of the brains behind the idea – ‘HeritageRoots’ – puts it:

When you’re no longer hunting, traditions about plants and animals and how to live with them disappear. Through representing lost knowledge in a technological form, young people here may have a new and productive way of working with elders in the Territory.

That speaker is Tod Swanson, an Associate Professor at Arizona State University’s School of Historical, Philosophical, and Religious Studies.

His HeritageRoots colleague and primary leader in technology choice and implementation, Dmitriy Babichenko, adds:

A lot of the culture that’s disappearing, both the culture that explains social structures, people’s relationships to their environments, certain linguistic concepts, but also the connection of culture to biodiversity, is contained in stories that people tell.

The complex, but dwindling, world of life in the forest

The context for Swanson and Babichenko’s work is helping to study and preserve the culture of the inhabitants of the ‘Oriente’ – Ecuador’s slice of the mighty Amazonian Biome.

Due to its unique location at the juncture of the Andes and the Amazon, Oriente soil is enriched by black volcanic ash coming down in the rivers.

That means that although it only accounts for 2% of the Amazon, Ecuador’s 135,000 square kilometers of rainforest is one of the most biodiverse areas on the globe.

It’s also home to the Amazonian Kichwas—a group of indigenous people sharing a common language, Kichwa.

Despite there being 100,000 speakers of the language, the pressures of modernity, and young people finding work to support traditional sources of income, like hunting and fishing, means that the culture and lore that previously sustained the community could soon vanish.

But key to the Kichwa worldview is the idea of ‘story.

This doesn’t mean our Western ideas of very bounded narratives with clear beginnings and ends.

Says Swanson:

The forest is experienced from many different directions and via different parts of the narrative. Typically, these are not told from beginning to end in one setting, but come in different pieces and at different occasions.

Thus, Kichwas live in a rich tapestry of tales that impart both practical/folk wisdom, which crystallizes their unique relationship to the forest and the animals that inhabit it with them.

Kichwa tales also include many real-world objects, like the tools and eating utensils humans use – even very specific parts of the trail or natural environment.

Swanson and Babichenko’s insight was that if information could be properly stored, which reflected the rich interconnections between the elements in a story, and the place it was told, the Kichwa experience and heritage could be modelled – and in a very rich way.

Graphs – a natural way to represent interconnectedness

Babichenko – Clinical Associate Professor at the Department of Informatics and Network Systems in the school of Computing and Information at the University of Pittsburgh – had previous exposure to the idea of graph-based knowledge graphs – a computing paradigm based on modelling a system based on the entities or elements in it, connected by the relationships these have with one another.

He says:

The problem with using relational databases is that entities are represented as tables. And tables assume that every entity has the same attributes describing it – but in our case, every entity could potentially have a very different set of attributes describing it. 

Different animals can produce different utterances; different media and plants could have different properties, and there’s also a virtually infinite number of connections which is impossible to replicate in a document database or in a relational database. 

A graph database is a natural choice whenever you’re trying to represent semantic relationships between objects, so we decided to work with student volunteers to start capturing Kichwa mythology in a graph.

Swanson and his collaborators input a wide range of stories from native speakers, who are encouraged to explain as much as possible about what’s going in each narrative and what the significance of the characters in them are.

They use Neo4j as the primary database for this, a Python application that then ingests all this information.

Babichenko then created the system with a web-based interface using Flask as a very light-weight web server, with a front end written in the HTML5 React framework.  D3.js is also being used to power a data visualization library, while extensive use of ChatGPT 3.5 is also central to modelling the content.

The result is a system that dynamically links a new story element to all its other instances – creating a growing web of Kichwa legends and wisdom.

Babichenko says:

We’ve created a pre-defined ontology of relationships, where the narrative itself becomes a node, and everything is connected to that node. So, as a narrative gets uploaded into the system, the model looks for all the animals, plants, human characters, objects, and geographic locations that are mentioned, and connects to those.

A move to immersion

The user interface also allows the upload of visual assets. As different stories associated with plants, birds, animals, and insects come online, short videos of these elements can also be linked.

In addition, more and more 3D scans of story elements, like (human-derived) bowls, masks, baskets, and hunting equipment, as well as natural objects like vegetation or Toucans, which are also being embedded in the platform.

This is starting to give the system real power, says the pair, as if there is a dialog that’s mentioned in a narrative, or are there actions that are associated with an animal or a plant or a human archetype, animation sequences to bring it even more to life can be generated. 

He says:

If I add a 3D model that represents a Toucan and the Toucan says something in the narrative, I can actually algorithmically animate the bird to say those things, or to fly or to sit on the branch.

But there’s an even more ambitious step beyond that: HeritageRoots will soon be hooked up to videogame engines like Unity or Unreal.

The vision here, says Babichenko, is to enable visitors to the Kichwa universe to literally step inside it.

He says:

One of the other reasons we picked Neo4j is because it does a really great job of outputting query results as JSON, Javascript object notation. 

That means we can build RESTful API endpoints for these 3D game engines that can kick off a procedural generation to create a simulated environment in which the narrative is occurring.

Swanson points out that the value here is that while text-based interfaces are fine, to engage young Kichwa learners growing up as part of the diaspora and now living in the world of Baldur 3, ways to draw them back to learn about their heritage need to be explored as much as possible – including as potential VR walkthroughs.

For Babichenko:

Soon, you’ll be able to ‘walk’ around the world of the Kichwa, where interacting with a particular plant or picking up a bowl would trigger the next narrative.

The more narratives and the more media HeritageRoots can bring into the system, the more immersive the system becomes.

But Babichenko wonders if there could be a lesson here for museums, which could avoid having to spend hundreds of thousands of dollars building a detailed geographic and historically accurate environment, and instead use tech to build accurate representations students and visitors could lose themselves in instead.

Capturing the moments of beauty 

For now, these are ambitions for the future.

Right now, HeritageRoots consists of a small but growing system of 12,000 nodes and 30,000 edges.

Next steps include launch of a public taster version on one of the Pittsburgh University servers, and a Summer of asking student volunteers to build out the system with publicly available, out-of-copyright narratives.

He concludes:

We’re also having conversations with lots of museums; once we launch, I think it’s going to start growing exponentially very quickly.

Adds Swanson:

I’m hoping we capture moments of the beauty and pathos of the Kichwa way of living socially with nature that’s deeply moving, and which is under threat because of the rapid change.

I also hope that our grandchildren can go into these virtual worlds and hear the voices of people from the past speaking their native language in the way that it was spoken when people were living day-to-day with these plants and animals.

If we can do that, that world will come alive and emotionally move people, and so hopefully motivate them to learn more, spend time in their home territories, to visit, to engage, and support their vanishing languages, cultures and home environments.