Enrich Transcription/Translation Data with Detected Language and Original Text

[OkGoDoIt]

Closes #363

This pull request enhances the data provided in translation stream to let Third-Party Applications (TPAs) Now the original language of a segment as well as the original text. This will allow apps to only translate text that is actually in a different language.

Problem Solved:

Previously, TPAs subscribing to translation streams couldn't easily determine if a translation actually occurred (vs. source and target languages being the same) or what the originally detected language was if it differed from the requested source language. Similarly, transcription streams didn't report the language detected by the underlying speech service. This made it harder for TPAs to build robust and context-aware features.

Changes Implemented:

1. SDK Interface Updates (packages/sdk):

   • Added optional detectedLanguage?: string and originalText?: string fields to TranslationData.
   • These additions are backward compatible as they are optional.

2. Cloud Service Logic Updates (packages/cloud):

   • Modified TranscriptionService (setupRecognitionHandlersForInstance method):
     • Since the azure api does not return the detected language, we determine it via matching the original text versus the translated text
     • Populates the new detectedLanguage field in TranslationData object.
     • Captures the original event.result.text and populates the originalText field in TranslationData.

Backward Compatibility:

These changes are fully backward compatible. Existing TPAs consuming TranslationData will continue to work without modification, as the new fields (detectedLanguage, originalText) are optional and will simply be ignored if not used by the TPA.

Testing:

I have tested this manually in my development environment. At the moment there are not any automated tests.

[netlify]

👷 Deploy request for augmentos-appstore pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	771f912

[netlify]

👷 Deploy request for augmentos-developer-portal pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	771f912

[OkGoDoIt]

This is ready for review

[CaydenPierce]

Looking great. Let's get translation using this end to end and then merge it up once we have a TPA (translation) that is successfully using.