GSoC 2022 :: Final Report

This post will briefly cover:

• Learnings
• Tasks done and blogs / weekly updates

Offline Voice Agent for AGL

This report is a requirement of the Final Evaluations phase of Google Summer of Code program. I worked with the Linux Foundation, under Automotive Grade Linux on the project to An Offline voice-agent for Automotive Grade Linux.

The proposal for the project can be found here.

The Linux Foundation is the nonprofit consortium dedicated to fostering the growth of Linux. Automotive Grade Linux is an open-source project hosted by The Linux Foundation that is building an open operating system and framework for automotive applications.

Project Deliverables

• Create and integrate a new Layer named meta-offline-voice-agent with Automotive Grade Linux (AGL) layers for the offline voice agent, using Vosk-API as the offline speech recognition engine.
• Create custom recipes for all unmet dependencies and libraries required by the Vosk API and Server to work.
• Create a PyQt based basic application to test offline voice recognition using vosk-server.
• Create a gRPC-based Python server and Flutter Client Application for integration with AGL.
• [WIP] Improvements and Integration of the created basic demo application for the voice agent. Possibility of testing on real hardware.
• Ultimately documenting the whole implementation methodology for the developer community to develop similar applications for AGL.

Community Bonding

Owing to timezone differences, my mentor and I scheduled the right time and medium for contact: weekly Zoom Calls with other GSoC participants for all updates and roadblocks, and constant communication over Telegram. Further refinement of objectives and goals were set to ensure that the proposed plan of action for the development of meta-offline-voice-agent was in line with the priorities of the project. I started engaging actively within the AGL community by regularly participating in the Weekly Developer Call. I have also regularly updated personal blog to log progress for future reference and help future Google Summer of Code applicants and aspiring student developers alike. Since I was already acquainted with the AGL community, I could start with setting up the project and coding-phase a little sooner, during the end of community bonding period itself.

Personal Roadblock

Starting from Week 9 upto Week 15 of GSoC’22, I had to take a break because of my relocation to Italy from India and all related preperations to commence my Master’s of Science studies in Robotics and get used to living in a new country. For the same reason, my project deadline was extended.

Work Done

Weekly Updates

All of the progress is well documented in the form of weekly updates over the agl-dev-community mailing list and personal blog :

• Community Bonding Period ==> AGL mailing list | blog
• Week 1 and 2 ==> AGL mailing list | blog
• Week 3 ==> AGL mailing list | blog
• Week 4 ==> AGL mailing list | blog
• Week 5 ==> AGL mailing list | blog
• Week 6 ==> AGL mailing list | blog
• Week 7 ==> AGL mailing list
• Week 8 ==> AGL mailing list
• Week 9 to 15 ==> Break and deadline extension due to relocation to Italy from India for Master’s studies.
• Week 16 ==> AGL mailing list
• Week 17 ==> AGL mailing list
• Week 18 ==> AGL mailing list
• Week 19, 20 ==> AGL mailing list
• Week 21, 22 ==> AGL mailing list

Commits / Change Requests

AGL Minor Documentation Fixes

• Gerrit: #27370 (and #27339) : Fixed indentation error for a sub-bullet point
• Gerrit: #27537 (and 27520) : Fixed internal link error
• Gerrit: #27575 : Fixed quote errors in echo expressions

meta-offline-voice-agent Layer

• Gerrit: #27794 : meta-offline-voice-agent: Enable layer for offline voice recognition
  • This enables offline voice recognition in AGL
  • Can successfully run vosk API and vosk websocket server

voice-agent demo app

• GitHub

Final Comments

Future Work

• Improve and finesse basic demo voice-agent app for AGL, create docs and app-integration with AGL

Challenges

• Becoming proficient with AGL developer tools, especially Gerrit has been quite a worthwhile effort.
• Having to relocate to a new country in between GSoC, and then keeping up to the academic expectations of Master’s degree towards the end of GSoC period made it slightly difficult for me to manage all things together, but due to mutual understanding and support from the mentors, I was able to accommodate some new changes in the deliverables and keep the project work going upto the completion of a working demo.

What did I learn?

• I was able to learn about the Yocto project, and creating layers and recipes for our specific application

• I learnt basics of creating aplications using Qt, PyQt and Flutter.

• Learnt about gRPC and GStreamer

• I also learned a lot more about Automotive Grade Linux organization especially their developer tools, workflow, and developmental pipeline.

• My technical writing skills have definitely become better and have led me to contribute more effectively and robustly to academic research projects.

Post GSoC

• I plan to maintain and add to the Automotive Grade Linux’s repositories for the foreseeable future and work on continually improving it.

• I will keep contributing and help other new contributors to explore and learn projects in this organization.

Acknowledgements

I would like to express my gratitude to my mentors Jan-Simon Möller and Walt Miner for mentoring me through GSoC and also to the whole AGL community for being so inclusive and friendly.

It has been one of the best professional experience that I undertook this year. Learning about new tools and technologies, and working on an open source project along with experienced developers from all over the world has been a really great experience for me. I would love to continue working with AGL, improving my project, and whatever I feel that I can contribute to.