How Alida is Leading Research Innovation with OpsGuru’s GenAI Guidance
Start Your Generative AI Journey
Background
The accelerating use of Generative AI (GenAI) is creating many new opportunities to improve business processes of all kinds. One of these opportunities is how to interpret product feedback from thousands of customers at a time, providing immediate feedback and actionable insights for global brand leaders.
The challenge of accurately interpreting feedback is critical for Alida, a global leader in experience management and community-oriented research for Fortune 500 clients. Alida has built a reputation for high-quality community feedback with incredible response rates (as high as 30%). To do this, they employ several survey methods and analysis techniques, including sentiment analysis and topic summarization.
Traditional methods, while useful, fall short in terms of speed, accuracy, and labour intensity. The advent of GenAI presented a possible solution to improve their service offerings. Alida was curious to see if AI could work in this field to produce accurate results better than those from current methods yet concerned about how to proceed while protecting the data privacy of its clients and their customers' data. That’s where OpsGuru came in.
Innovation Through Partnership
OpsGuru, a Carbon60 Company, is a Premier Amazon Web Services (AWS) Cloud Consulting Partner, with over 500 successfully implemented projects focused on cloud assessments, acceleration services, and productivity solutions. With a growing emphasis on AI, particularly GenAI, OpsGuru was a perfect fit to help Alida, having worked with them on several other cloud projects with glowing results since 2022.
OpsGuru was a natural choice to lead this project, having previously guided Alida through an AWS Cloud Data Migration project. A key part of the overall solution is early use of AWS’s Bedrock platform, a managed service that allows customers to leverage GenAI models from various providers via API without needing dedicated hardware or specific contracts with 3rd party vendors.
The Challenge
Interpreting Uniquely Human Experiences
The challenge centers around how market researchers perform qualitative analysis, and how difficult it is to do. Quantitative analysis is easy to perform, as the questions are structured into yes/no queries or numerical ranking (one to five). These types of results are easy for researchers to tabulate and summarize using statistical analysis software.
But qualitative analysis, while incredibly valuable, is much harder to do. Qualitative analysis involves the use of open-ended questions where responses can be sentences long and include multiple topics, concepts, and sentiments.
Researchers analyze data by closely examining responses to summarize topics, meanings, and sentiments, a process known as coding (distinct from programming). This tedious task demands language-savvy individuals with quick analytical skills and cultural knowledge. Alida's research, sometimes involving tens of thousands of respondents, can take weeks to analyze.
While software exists to analyze open-ended responses, it only looks for keywords. It can’t pull sentiment or meaning out of sentences and has a hard time summarizing the data. Even when software is used, people still have to analyze the results the software provides.
GenAI seemed to provide a potential solution to this, but a further challenge is that Alida’s global clients operate in highly complex regulatory environments where the protection of their customer’s data is of paramount importance. There are risks associated with using AI tools, including potential data leakage, i.e., ingestion of user data to further train AI models, which some GenAI models do. Thus, any solution requires doing the job safely and ethically.
Our Solution
OpsGuru Recommended AWS Bedrock and Anthropic Claude
At Alida’s request, OpsGuru developed the GenAI proof of concepts to evaluate efficacy, reliability, speed, and risks for sentiment analysis and topic summarization. The desired outcomes were to see if the tests could unearth nuances in patterns and sentiments with unstructured data and whether the results could equal or surpass human analysis and output, without putting data at risk.
OpsGuru and Alida chose Amazon Bedrock, in collaboration with AWS, for its comprehensive benefits: it's a managed service offering access to high-performance AI models via an API for easy integration, with robust privacy and data security features. Its data storage in certified secure centers, with regional segmentation, supports global data privacy needs, underlining Alida's commitment to data protection.
The selected model, Anthropic Claude, is a constitutional, ethics-focused model that uses principles for output evaluation, bypassing human bias. Designed for complex reasoning and technical tasks, it surpasses ChatGPT with abilities to process uploaded files, handle more words, and access post-2021 data.
For Alida’s purposes, Anthropic Claude was the best model to use and was chosen after OpsGuru’s Clear Path Forward for Generative AI. A readiness assessment, which included a two-week technical deep-dive, was conducted and a thorough analysis of Alida’s goals and technical and business challenges. This led to the recommendation of the type of AI model to use, as well as the proof-of-concept plan.
OpsGuru’s experience with AI models was vital in helping tune the solution specifically for superior performance in topic reduction and summarization.
The Result
Greater Accuracy, Faster Results
Since the proof of concepts have been running, Alida has realized incredibly positive results so far. The results were compared using a test bed of highly anonymized data with human analysis, and computer-based keyword analysis.
What the results have shown is a vast improvement in both sentiment analysis and topic summarization accuracy when compared to Alida’s previous attempts using computer keyword analysis and superior results to existing human-based efforts.
There are immediate tangible benefits for their clients, where automated reports are instantly usable, without requiring additional human analysis to interpret the results.
And the GenAI process is fast. It can process 10,000 responses in a few minutes and can process up to 100,000 responses in just a few hours. “It’s really a quantum leap forward in terms of performing sentiment analysis and topic reduction at scale,” said Finneran.
Lessons Learned: Worth Experimenting with AI
Overall, this was an experiment worth trying for Alida. “With OpsGuru’s help, we’ve learned a lot about just how capable AI is going to be in market research analysis. And our early experiments have given us a leg up on the competition,” Finneran remarked. “This shows us that this can be done in a way that produces results and is safe with our client’s data.”
Future Directions: New AI Features Quarterly
Alida’s AI ambitions are not slowing down, with a goal to deliver one new AI-based feature per quarter. Some of their next steps will be to integrate real-time translations into the responses so they can pull in customer experience insights from around the world for their global clients. With OpsGuru’s AI expertise and guidance, Alida can innovate faster.
Conclusion
The early success shows how working with OpsGuru on AI projects can help clients solve tough problems and progress in a time of fast tech growth. As Joseph Finneran aptly put it, "Innovate or die. That’s the way forward." With its bold foray into GenAI, Alida, supported by OpsGuru and Amazon Bedrock, is not just surviving—it's thriving, setting new standards for the industry, and opening up new possibilities for the future of experience management research.