Highlights of Global AI Dev Day in New York

Hello readers, I attended the 10th AI Dev Day at Microsoft Office, New York on October 26th, 2023, and in this blog, I’ll be sharing the different insights that I got from the event.

The event was hosted by AiCamp at Microsoft, NYC. In addition to the amazing hot chocolate, free pizza, and the raffle giveaway, the talk by Ryan Michael, VP of Engineering at Kaskada, was very informative, and new for me. I didn’t have much idea about applications of generative AI and machine learning in the real-time data processing space before.

“Kaskada is a next-generation streaming engine that connects AI models to real-time & historical data.”

Ryan gave a lot of insights on what LLM’s are currently doing and how they can be powered to use their ability on Real-time data streams.

• The best way to use an LLM right now is either Prompt Engineering, or Few-Shot and RAG (Retrieval Augmented Generation). The next step towards using these LLM’s is applying them to Real-Time data-aware AI.

• Ryan went on to explain how Real-Time Proactive AI’s flow will be. First, the AI will observe and Watch the events happening in real-time. Within that real-time events and data, a specific event will Trigger the call to an LLM, and the LLM’s response will be used to further update values. Some of these responses or events will trigger an action and the AI will Act on that event. These actions and triggers update values or trigger further workflows based on the results of different actions and updated values. The model is then made to learn from the past events-actions-results using RAG, few-shot learning, fine tuning and other traditional approaches for further enhancement.

• Here he explains different ways in which triggers can be handled. One way is the “Lazy” way which basically means that we have a bunch of different queries and we have a static data, on which these dynamic queries can be execute. This is the traditional ‘lazy’ approach which is used in most of the traditional applications of these systems because they were originally built to be used that way, but in the ‘eager’ method of computation, we actually keep on appending data and updating our data and we have a single query on which we pass on this data and get an answer for it. So instead of having static data, we have a static query and this method is better optimised for decision making because in decision making, every time a different set of data can be sent to the same query and results can be extracted.

• Next Ryan discussed about how Kaskada can be easily pip installed and used to process real-time data streams. This process is very well explained in their documentation.

• Then he presented a Slack bot named BeepGPT developed by Kaskada, implementing Kaskada’s ability and efficiency to perform computations on time based data, such as bank transactions, or chat messages, etc.

• BeepGPT is basically a slack bot designed to notify users of a specific channel whenever it decides that a specific conversation is important for that user. It is basically based on a fine tuned curie model from openai, trained to predict the next person in a specific conversation. So, in real time, it predicts the next person to chat, and then pings that person with that specific conversation’s url to tell him that there’s a chat that might be interesting to you. To know more about BeepGPT, it’s official github repo can be referred.

I found the session to be extremely informative and thought provoking. I got to know about this other set of applications of LLM’s and AI which is processing data in real-time. The applications for such an application can range from banks processing payments to processing messages in a specific chat and much more.

Note: The pictures used above are from the session or from the youtube video.

And of course,

I look forward to the exciting discussions, collaborations, and discoveries that await us. Stay tuned for more insightful content, and let's embark on this journey together!

Connect with me on LinkedIn