Yes ladies and gentleman, a post about developing with AI!

If you're team is looking to incorporate an LLM into your services, the first challenge to overcome is how to do so in a cost effective way.

Chances are, your business is already focused on a specific product domain, with resources targeted towards building that solution. This already is going to point you towards finding an off the shelf solution to integrate with an API.

With your flavor of LLM picked, the next set of challenges center around getting it to respond to questions in a way that meaningfully provides answers from your business data. LLM's need to be informed on how to respond to requests, what data to utilize when considering their answers, and even what to do if they're tempted to guess.

The way forward is through prompt engineering, with the help of Retrieval-Augmented Generation

Retrieval-Augmented Generation

The simplified procedure for RAG goes as follows:

1. Request is made to your app with the message "How many Tex-Mex Restaurants are in Dallas?"
2. Your application gathers context. For example, we may make a query to our DB for a summary of all restaurants in the area.
3. We'll provide a summary of the context and instructions to the LLM with a prompt.
4. We send along the response to the user.

That's an overly simplified walk through, but it should already get you thinking about the details involved in those steps depending on your use case.

Another benefit to this is that requests to an API are not inherently stateful. The chat window of an AI app will remember our previous messages. But my API request to that third party does not automatically. We have to store and retrieve that context.

AI Agents

It's worth noting that step 2 may even require an LLM to parse the question and then interact with an API to gather data. There's a fair amount of complexity to still parse in developing these solutions. This is where you may be leaning on an AI Agent. An agent is an LLM that will parse a result and determine if a tool is required, such as pinging your internal APIs.

Prompt Engineering is emerging as a role and craft all of its own, and there are many nuances to doing it well.

LangChain

The workflow is already so common that there's a framework at the ready to spin up and take care of the heavy lifting for you. LangChain (stylized as 🦜⛓️‍💥) is just that tool.

For a hands on experience building a RAG application on rails, their docs on building a chatbot are a good starting place.

For a more complex agentive tool, LangGraph opens up the hood on LangChain for more control and plays nicely with LangChain when needed.

Listen on Youtube

Gather round, and listen to this tale...

Just a bit of noodling between practicing longer pieces.

It's grey out this time of year. But behind the clouds, there's always a blue sky. 🌤️

Polymorphism! A sturdy pillar in the foundation of Object Oriented Programming. At it's simplest, it's the ability to change the implementation of specific methods on derived classes.

At face value, that could mean entirely rewriting the method. But what if we want a bit more nuance? What if we want to extend instead of replace the method entirely.

I'll continue on my Vehicle example from my previous post on polymorphism, this time in Python:

from abc import ABC

class Vehicle(ABC):
    def __init__(self, color: str):
    
        if not color:
            raise ValueError("Color string cannot be null")
            
        self._passengers = []
        self.color = color

    def load_passenger(self, passenger: str):
        # Logic to load passenger

    def move(self):
        # Some default code for moving
        print("Moving 1 mile North")

I've created an Abstract Base Class that serves as a starting point for any derived classes. Within it, I've defined a method move() that moves the vehicle North by 1 mile. All children will have this class available automatically.

Now, if I want to override that method, it's as simple as declaring a method of the same name in my child classes:

class Car(Vehicle):
    def move(self):
        print("Driving 1 mile North")


class Boat(Vehicle):
    def move(self):
        print("Sailing 1 mile North")

In the case that I want to extend the functionality, we can use Super() to do so:

class Car(Vehicle):
    def move(self):
        super().move()
        print("Pedal to metal!")


class Boat(Vehicle):
    def move(self):
        super().move()
        print("Raising the sail!")

The benefit here is I can pass all the same arguments I'm receiving in the method call on either child instance to the default implementation in the parent. They can then be used in my own custom implementation in the child class.

car = Car()
car.move()
# Moving 1 mile North
# Pedal to metal!

Listen on Youtube

'Scuse me while I disappear~ 🌫️

Getting ready to move neighborhoods next month. The current place is just walking distance from a beautiful trail. So I'm savoring it while we're still here!

Listen on Youtube

My first swing at chord melody! Love this tune even more on guitar.

The veil is thin. 👻

Squeaked in one Inktober drawing this year! Very much directly inspired by the energetic ink work of Violaine Briat's Lil' Dee.

So much of JavaScript/TypeScript is massaging data returned from an endpoint through JSON. TypeScript has the lovely ability to type the objects and their properties that come through.

While Python is not as strongly typed as TypeScript, we have this benefit built in to the type hinting system.

It's easier shown than explained:

from typing import Union, TypedDict
from datetime import datetime


class Concert(TypedDict):
    """
    Type Dict for concert dictionaries.
    """

    id: str
    price: int
    artist: str
    show_time: Union[str, datetime]

All pretty straightforward. We're instantiating a class, inheriting from the TypedDict base class. Then we set our expected properties as values on that class.

It's ideal to store a class like this in its own types directory in your project.

A couple of nice ways to use this:

First, you can use this in your methods where you are expecting to receive this dictionary as an argument:

def get_concert_ticket_details(
        self, concert: UnitDict = None
    ) -> tuple(list[str], set[str]):
    // Do work

You can also directly create a dictionary from this class through instantiation.

concert = Concert({
    "id": "28",
    "price": 50,
    "artist": "Prince",
    "show_time": show_time
})

The benefit of both is, of course, the suggestion in your editor letting you know that a property does not match the expected shape.

More details on Python typing in this previous post. Thorough details available in the official docs.

Listen on Youtube

Today I learned that this jazz standard is named after margarine. Yum!

We have bobcats and coyotes on the other side of the lake near our home. You can hear them at night. Every now and then, I see one looking back at me 🐺

I remember the day I logged onto ye olde facebook after a layout change. A few groans later, what really blew me away was the immediacy of my comments on friends' posts. I was used to having to wait for a page refresh, but not anymore! Once I hit submit, I could see my comment right on the page with no wait time.

That's the power of optimistic UI. Web applications maintain a high level of engagement and native feel by utilizing this pattern. While making an update to the page, the actual form submission is being sent off to the server in the background. Since this is more than likely going to succeed, it's safe to update the UI on the page.

There are a few libraries that make this process a breeze in React. One option is Vercel's SWR, a React hook for data fetching.

Data Fetching

Say I have a component rendering data about several cats. At the top of my React component, I'll fetch the data with the useSWR hook:

const {data, error, isLoading, mutate} = useSWR(['cats', queryArguments], () => fetchCats(args));

If your familiar with TanStack Query (formerly React Query), this will look very familiar. (See my previous post on data fetching in React with TanStack Query for a comparison.)

To the hook, we pass our key which will identify this result in the cache, then the function where we are fetching our data (a server action in Next), and optionally some options (left out above.)

That returns to us our data from the fetch, errors if failed, and the current loading state. I'm also extracting a bound mutate method for when we want to revalidate the cache. We'll get to that in a moment.

useSWRMutation

Now that we have data, let's modify it. Next, I'm going to make use of the useSWRMutation hook to create a method for changing our data:

const {mutate: insertCatMutation} = useMutation([`cats`, queryArguments], () => fetchCats(args)), {
        optimisticData: [generateNewCat(), ...(data],
        rollbackOnError: true,
        revalidate: true
    });

Note that I'm using the same key to signal that this pertains to the same set of data in our cache.

As you can see, we have an option that we can pass in for populating the cache with our optimistic data. Here, I've provided an array that manually adds the new item through the function generateNewCat(). This will add my new cat data to the front of the array and will show on the page immediately.

I can then use the mutate function in any of my handlers:

const {error: insertError} = await insertCatMutation(generateNewCat());

Bound Mutate Function

Another way of accomplishing this is with the mutate method that we get from useSWR. The main benefit is we now get to pass in options when calling the mutate method.

const handleDeleteCat = async (id) => {
    try {
        // Call delete server action
        deleteCat({id});
        
        // Mutate the cache
        await mutate(() => fetchCats(queryArguments), {
            // We can also pass a function to `optimistiData`
            // removeCat will return the current cat data after removing the targeted cat data
            optimisticData: () => removeCat(id),
            rollbackOnError: true,
            revalidate: true
        })
    } catch (e) {
        // Here we'll want to manually handle errors
        handleError(e);
    }
}

This is advantageous in situations like deletion, where we want to sequentially pass in the current piece of data targeted for removal. That can then be passed both to our server action and updated optimistically through SWR.

For even more context on using optimistic UI, you can find a great example in the SWR docs

Listen on Youtube

Finishing out this bossa 🏝️

To be someone's pumpkin...

This week I'm synthesizing notes from Michael Pogrebinsky's course on Software Architecture and Design. If you like what you read below, you'll love the course!

When designing a system, especially at higher scales of complexity, it's vital to take the carpenters' motto to heart: "Measure twice, cut once."

Smaller scale projects, such as feature requests for an already existing system, have very clear restraints. The programming language, platform, and infrastructure of the project have already been laid. It only takes a few questions to get clear on what needs to happen when adding a widget to an existing app.

Larger solutions, however, can be daunting with the sheer vastness of options. Any software choice can solve any problem. And, in spite of what the discourse on forums may lead you to believe, there isn't one correct solution for any problem.

So how do you narrow down your choices? Through requirement gathering!

Requirements As Part of the Solution

A paradigm shift for anyone pivoting from a small scale environment to thinking broadly is understanding that question asking is part of the solution.

The client, sometimes not technical, will have an idea of the problem they want to solve, but will be unclear on the solution. Even if they have an idea of what solution they would like to see implemented, you as the technical authority in the room have to ask clarifying questions to find out if their suggestion will work in your system.

Types of Requirements

To help ensure time is spent asking the right questions, it's helpful to know that there are three types of requirements you can gather:

• Functional Requirements: Features of the system. Think inputs and outputs. Listen for "the system must do this"
  • Generally, these do not determine the architecture. Any architecture can solve any problem.
• Quality Attributes: Non-functional requirements. Deals with performance of the application. Listen for "the system must have".
  • Ex: Scalability, availability, reliability, performance, security. A more comprehensive list here.
  • These dictate the software architecture of our system.
• System Constraints: Limits and boundaries.
  • Ex: Time, staffing, resources. Can also drive architecture design.

Example: CD Dad

Say you're working with a client that is looking to build out an e-commerce platform for musicians. A possible portion of requirements may include the following:

"CD Dad will host albums from independent musicians. When a customer purchases an album, the customer will receive a digital download of the album and the musician will be paid."

So far we've heard the Functional requirements. With given inputs, the system hosts the music. When a customer submits a purchase, they receive the digital downloads.

"Processing uploads should be take no longer than 5 minutes. When an album is purchased, a link should be made available immediately through email."

The statement above relates to performance of the app, so this is a quality attribute.

"The system should support mp3, wav, and aif file formats. A team of a dozen full time engineers will be responsible for maintaining the system."

Now we're talking file formats and engineering support, so we're looking at System Constraints.

The Joy of Constraints

For many engineers, an ideal world is one without constraints. However, limitation breeds creativity. A limit on resources, hands on deck, and time are what enables us to ship code regularly.

In the event that any of this information is missing in your understanding of a system, it's an opportunity to gather more info and clarify what's being built. Doing so will help clear the fog for the next best step.