Gaya is revolutionizing the way insurance agents work. We use AI technology to fundamentally improve the way insurance professionals quote, re-market, and sell, all while enabling them to grow their businesses. The first challenge we’re tackling is that of tedious form-filling. As such, we are thrilled to introduce the Super Clipboard, our innovative browser extension that's set to redefine copy and paste.
Transforming Workflows
Our Super Clipboard is an AI-powered product designed to copy information from any website or physical document (PDFs, images, handwritten notes), intelligently mapping the right entities onto the fields of any web form. Our product is the result of a complex and comprehensive engineering effort, leveraging advanced technologies like large language models (LLMs), computer vision, and vector embeddings.
As we continually track user interactions, a powerful user network effect is set in motion. Every interaction contributes to this effect, strengthening our strategic moat and enabling the creation of automation recipes. The power of this network effect, coupled with machine learning, allows us to generate a robust library of these recipes over time. This library will drive our Super Clipboard to recommend the most efficient workflows to our users.
Traditionally, scraping activity has been considered to be unreliable given that scripts will break as soon as there is a UI change or if some of the elements identifiers on a webpage (commonly referred to as selectors) change. Our Super Clipboard is able to adapt to UI and element changes by leveraging LLMs. We feed relevant sections of an HTML object to a fine-tuned LLM model to detect changes on web pages and adapt accordingly. To learn more about our Super Clipboard and how it's set to redefine productivity, visit gaya.ai .
Our Engineering Challenges
The Super Clipboard's development presents a series of engineering challenges, some of which can be solved with AI solutions, others requiring us to build sets of unique solutions. Here are just ten of the challenges we're addressing:
- Handling Multiple Entities: Webpages and documents frequently house multiple instances of similar data, such as a form that includes multiple vehicles, each with similar fields (make, model, and VIN number). The task of accurately extracting this information and mapping it to the correct object demands highly sophisticated AI models. The challenge lies in ensuring that our AI understands the context, can differentiate between similar entities, and associates the right data with each entity.
- Dealing with Dynamic Webpage Selectors: Some websites use dynamic selectors that shift with every page refresh, which can disrupt the data extraction process. To circumvent this, we're building methods to identify and trace these dynamic selectors. This involves fine-tuning an LLM model that can detect changes in any relevant website sections and respond appropriately.
- Navigating iFrames: iFrames, or webpages within webpages, add another dimension of complexity to webpage scraping. Our product needs to penetrate these embedded elements to extract and handle the data they contain. This entails leveraging advanced programming techniques capable of interacting with these nested web environments.
- Managing Dependent Dropdowns: Certain forms have dropdown menus that rely on previous inputs and make API calls to retrieve associated values. For instance, a form may require you to select a state before choosing a city. This requires an intelligent system that can handle dependencies and execute asynchronous calls effectively.
- Mitigating Non-standard HTML Practices: Not all websites adhere to HTML best practices, which further complicates the extraction process. We're working on techniques that can navigate non-standard and inconsistent HTML structures, ensuring reliable data capture irrespective of the website's coding practices.
- Addressing Non-unique Selector IDs: Websites built using specific frameworks often lack unique selector IDs in their elements, adding another hurdle to data extraction. We're developing ways to identify elements reliably, even when unique IDs aren't available.
- Interpreting Inconsistent Dropdown Values: Different forms might represent similar values in dropdowns in varying ways. It's crucial for our model to interpret and translate these differences accurately, ensuring that equivalent information is recognized and handled consistently across diverse forms.
- Managing Differing Data Formats: Data can be presented in a multitude of formats across websites, such as dropdowns, checkboxes, radio buttons, toggles, or input fields. Our Super Clipboard must be capable of intelligently adapting to these format variations and correctly process the information it encounters.
- Mastering Event Listener Handling: To enable seamless pasting of data and ensure the proper functioning of element animations, we need to adeptly manage event listeners across different web pages. This challenge becomes especially critical when interacting with diverse frameworks like React, Angular, or Vue.js, where the triggering of events can vary significantly. Our tool is being developed to not only recognize these differing events but also respond to them appropriately, ensuring values are set correctly, and animations are triggered as intended.
- Addressing Combined vs. Split Fields: Details like dates, addresses, and phone numbers can be displayed in combined or split fields across different websites. Our tool needs to identify and seamlessly handle these variations, ensuring that information is extracted and inputted correctly, regardless of how it's formatted.
Despite these challenges, we're confident in our ability to navigate these complexities and deliver a powerful, transformative tool. Our approach has been to build our product with as much modularity as we can - we have a suite of building blocks to address each of the aforementioned challenges. These building blocks will enable us to continue innovating (and at a faster pace) in the insurance industry. In a future blog post, we’ll walk you through these building blocks and the ways in which we intend to leverage them to further increase the scope of our AI copilot.
To stay up-to-date on our progress and learn more about the Super Clipboard, check out gaya.ai