State of the art bilingual communication for live interactions

This Case Study is based on information provided by the AKT2I partners.

North Wales-based Haia, recent winners of the Best Startup in Wales 2023 and the Digital Startup of the Year, * provides a multilingual platform for online and hybrid events.

With previous events run on the platform by customers including MIT (Massachusetts Institute of Technology), the NHS and the Welsh Government, Haia is on a “thrilling ride to redefine the possibilities of online and hybrid experiences”.

Looking to enhance the speech-to-translated-text (STTT) feature on its online and hybrid platform, Haia entered into an AKT2I partnership with Bangor University (BU) whose academics have developed state of the art methodologies for Welsh, and other less-resourced languages.

The AKT2I project aimed to support Haia in accelerating the integration of STTT in its events platform using neural network models. This would enhance English and Welsh bilingual communication in real time for meetings, conferences, and other events where both languages are used concurrently. The knowledge gained would be transferable to other multilingual situations worldwide.

The requirement for improved STTT capabilities was driven by Haia’s growth strategy to become market leaders in Wales for bi-lingual online and hybrid events. The STTT functionality is estimated to increase revenue by approximately 20%.

AKT2I provided the opportunity to access skills and expertise whilst limiting commercial risk. It was not possible to understand the efficacy of neural network STTT on the Haia platform without implementing and testing within the platform itself; so the AKT2I programme allowed Haia to de-risk this proof-of-concept development activity whilst also accessing specialist expertise.

The project set out to provide machine based STTT from Welsh into English. The performance was analysed with respect to speed, efficiency and accuracy. Haia and BU hoped to benchmark the neural network driven STTT methods against other translation options and understand the requirements for further developments.

The project implemented and analysed two STTT models for use in the Haia platform. It was possible to train and test these models using three datasets. The results were analysed for baseline performance metrics, and model selection and improvements were made. Assumptions regarding the STTT model selection were tested and challenged and the project team was able to select the most appropriate model for Haia’s purposes and begin the process of wider integration.

A significant element of this project was to ascertain the viability of STTT models for use on the Haia platform. A computationally efficient and accurate STTT pipeline solution of speech to text (STT) and machine translation (MT) models were developed.

These developments could be sequenced in a pipeline to achieve full STTT within the Haia platform. The technology was observed by candidate users as successful for communicating meaning but still has opportunity for improvement.

The positive test results, and subsequent improvements in the models, demonstrated the suitability of this technology for this purpose. This was a highly positive result as it provides Haia with the confidence to progress this innovation – unique in the sector – and supports the company’s wider business strategy.

Welsh language – just the start of things to come?

It is anticipated that the innovation could be applicable to other language pairs. As a future development, this would position Haia in a unique position in the global events platform market.

These opportunities will be realised through continued partnership activities seeking to scale-up and mainstream the technology into the Haia development pipeline.

Wider language use brings positive societal impact

This project could deliver a significant societal impact by increasing the use of Welsh in hybrid events where a human interpreter is not feasible. It can contribute to normalising the use of Welsh in all manners of bilingual hybrid events and gatherings.

Haia is committed to continuing the collaboration, having recognised how Bangor University is able to provide knowledge, skills, and expertise that are otherwise unavailable on the open market. The partnership is now developing plans to apply for a grant to scale up the development and implementation off the STTT technology.

For further information about this project please contact post@haia.live or s.ghazzali@bangor.ac.uk