🤖👩🏽‍🍳 Cook Like A Bot: AI dinner parties

One year ago, I was 40 and didn’t know how to cook. So I set out on a journey to the most obvious solution: become a sous-chef to an AI. There have been highs and lows, yums and yucks, and lots of learning. Here’s the story so far, and what’s to come.

Intro

In June 2023, just one month into dating, my girlfriend Ava and I started hosting AI dinner parties in my small NYC apartment. AI would come up with a menu & recipes based on our inputs, which we cook & eat with friends, and rate the results. Intrigued by the personalization capabilities of Large Language Models (LLMs), we wanted a learning project that was fun, social, and delicious. Our initials are AI & ML, so it seemed like fate.

Recipe generation has been a popular LLM use case since the start of this AI boom. Initially I was running all kinds of ill-advised experiments, like juicing pulp “bread” for which I got roasted on Broadway. Fellow experimenters have also run other cool tests. Ava and I have kept going, pushing the boundaries, perhaps sometimes a bit overboard 🤪.

[Socially Inept, Sean Linehan, Shaan Puri, Sam Parr, WIRED]

Below is the evolution of our AI dinner parties. We describe our System, Results from 🇹🇭🇮🇹 Thaitalian / 🇯🇵🇲🇽 Japaxican / 🇬🇷🇰🇭 Greekbodian / 🦃🤝 Friendsgiving dinners, Lessons learned, and what’s Next.

System

The initial goals were to use AI to develop unique menus and recipes for social cooking nights, based on guest inputs, and intro AI to normies in a relatable way. ChatGPT helped brainstorm personalization variables like cuisine mashups, color focuses, feelings evoked, cooking techniques, holiday themes, ingredient textures, social causes, and even astrology signs. Here are the elements we’ve explored so far:

Menu

We started with cuisine fusions because recipes generated were less likely to already exist in the AI training data. For the first few dinners, guests listed cuisines they wanted to cook, and we manually selected the two most requested. That evolved to ChatGPT’s Code Interpreter generating subjective-but-reasonable Compatibility and Novelty scores from 0–100 for every possible fusion, then selecting MAX(Compatibility * Novelty ^ 2). Next we tried having set dishes (e.g. Thanksgiving traditional) modified based on each guest’s background and personality.

For dietary restrictions, we asked each guest about theirs and prompted AI to respect the limitations. Beyond vegan / keto / lactose-gluten-shellfish free, we collected limitations on ANIMALS (chicken / turkey / pork / octopus / squid / snails / clams / scallops / sea urchin), PLANTS (peanuts / mushrooms / avocado / seaweed / olives / bell peppers / lima beans / string beans), SEASONINGS (wasabi / ginger / caraway seeds / mint / cinnamon), self-aware semi-bans (“I try to stay away from cheese, but love it”), and probably-not-but-good-to-knows (poi, live fish, “big chunks of eggs”).

After the no-nos, we asked guests for flavors they liked. Menus were generated around including wishlist ingredients (shrimp, aloe, anchovies, coconut milk, masa, bok choy, thick noodles, eggplant), spices (cumin, paprika, dill, coco aminos, jalapeño, cardamom), and alcohols (aged rum, vodka, prosecco, margarita, classy gin, red wine). We also asked for mocktail vs cocktail prefs so the recipe would add booze only at the end.

AI also helped choose proteins. Guests voted amongst land (bison / pork / beef / rabbit), air (duck / chicken), and sea (fish / scallops / shellfish). Votes were ranked, restricted items were removed, and recipes were based around the remaining leaders. Plant proteins were also voted on, with at least one main course based on tofu / tempeh / seitan. These analyses required Code Interpreter to write a Python script: LLMs still suck at math.

Imagine expressing all these preferences to a restaurant, or hosting a party that respects every parameter. That’s hard for humans, but easy for AI. However, human judgment is still needed. Our first dinner party included a vegan friend, so AI made every course vegan. But sometimes you want to balance between needs of individuals and needs of groups. So we evolved the system to make sure that everyone had enough to eat, not every dish had to be for everyone, and it was clear what was what.

Recipes

Asking AI to generate recipes is easy, and average output is often fine. For a creative multi-course experience, more guidance is needed. We started by defining the courses: cocktail / appetizer / soup / salad / side / entree / dessert for most dinners, and turkey / mashed potatoes / gravy / stuffing / cranberry sauce / sweet potato / pumpkin pie for Friendsgiving. Over time we found that some courses benefitted from extra guidance like:

Cocktail: Make strong enough but not too strong, and include an interesting garnish on top.

Appetizer: Aim for finger food that can be eaten while standing and cooking. Cooking takes time and no one wants hangry guests.

Salad: Should have some texture (including something crunchy) and a tasty dressing.

AI also helped match proteins-courses-cuisines. A Chain of Thought approach worked best: ask for reasoning before recommendation. This helps AI “think” before it “answers.” Here’s an example of how AI worked out some logic, similar to how a human expert might think through it:

To make shopping easy, we avoided hard-to-find ingredients. After recipe generation, we asked AI to reflect on whether anything on the list would be rare in local large grocery stores. AI suggested which ingredients might need specialty stores, or offered alternatives. In rural areas this step could help with recipe practicality, and with enough notice most things can be ordered online.

We also had fun generating AI images of each dish in advance to tease our palate and compare with IRL final results (in Results section below). AI created image generation prompts in the style of professional food photography. Midjourney v5 was our initial tool, with great results, but was annoying to use within Discord. Later we switched to DALL-E 3 within ChatGPT: results were more variable, but it was easy to use inline. The power of integrated distribution! We’ve also tried generating images for each progressive recipe step (below). Consistency and realism are still highly variable, but this seems addressable soonish.

Vibes

Ambiance was considered as well. Initially we asked guests to vote on a theme color, as an easy first experiment. For the first dinner everyone came dressed in blue, we decorated the apartment blue, and AI even made the recipes blue. A few learnings:

1. If your friend doing decorations brings blue confetti poppers, you’ll still be finding sparklies in your furniture a year later.
2. There are cool organic options to make food blue like blueberries, butterfly pea flowers, and spirulina powder.
3. Blue spaghetti looks too much like Smurf intestines.

We then tried asking guests for theme ideas beyond just color. AI ordered the options in descending compatibility with the fusion menu, and even threw in some unsolicited advice.

Best: Mexico and Japan have extensive coastlines and many dishes that are inspired by the sea. The Beach theme can highlight the freshness of the ingredients, especially the fish in the salad. The relaxed vibe of a beach setting can also complement the fusion of two vibrant cuisines.

Worst: While a beloved story, The Wizard of Oz doesn’t have a clear link to the fusion of Mexican and Japanese cuisines. The theme might overshadow the uniqueness of the menu, making it the least compatible option.

Remember: The most important thing is to ensure that the chosen theme complements and doesn’t overshadow the culinary experience. The menu is the star of the show, and the theme should enhance, not detract from it.

Some fun Party City visits helped make the room pop. Going forward, we’ll have AI generate the theme, scan party supply websites, and compile a custom theme-on-demand order. Taping crepe paper to the ceiling will still be a human job until the Boston Dynamics robots can jump that high.

Our first events were back in the olden days before good quality AI music. So the original tactic was to manually curate a multi-hour YouTube playlist of Thai and Italian blues. That was fun, but a lot of work. So the next few parties just featured jazz and tropical house Spotify playlists, based on guest votes. More recently we’ve started generating custom AI soundtracks based on the theme, which will be covered in later writing.

Workplan

We had to make sure that this concept would physically work in my Manhattan apartment’s tiny open kitchen. Being insane, I had bought a professional restaurant station to double the available workspace. Even so, it was a super tight fit in both space and time for eight people to prep, cook, and plate unfamiliar dishes before we ate on spare camping chairs around an old glass desktop balanced on a coffee table.

For each recipe generated, we asked AI to list all kitchen equipment required and compare with our equipment inventory to identify gaps. We also tried (before multimodal input) describing the layout as workstations, and then visually diagraming it.

We’ve kept improving an AI-generated spreadsheet where the rows are minutes until T-0 serving, and the columns are workstation activity with instructions / equipment / ingredients. This mostly worked well, especially as we iterated a) analyzing each recipe separately and then combining, b) listing out what equipment was needed when, and c) adjusting recipe start times or cooking methods to be able to share workstations, equipment, and oven space.

This all was combined into a workplan displayed on the kitchen TV I mounted as, once again, an insane person. It did look cool, but was impractical: text was too small, and the timer was more stressful than helpful! Going forward, we’ll generate the workplan behind the scenes to check logistics feasibility, then give everyone their own untimed interactive recipe view on their phone screens.

Ingredients shopping also got some love, with custom AI shopping lists organized by store section make grocery trips easier. To minimize unnecessary buying, we asked AI to play a “logistics expert”. It organized all the ingredients into three categories: what the House already had, what the guest needed to Buy, and what they could Borrow from another guest who was using a larger amount of the same ingredient and could Lend some of theirs. This worked better than we thought and minimized waste.

Agents

We’ve started using bots as negotiating agents to decide who will cook what. We asked guests for their preferences on each dish: 💯 Would love, ⚖️ Either way, or 👎 Rather not. Folks also set their bots’ negotiating style, and gave personal characteristics for dish customization. There were a variety of styles, including random “My Octopus Teacher” mentions, “secretly Machiavellian” pursuit of cranberry sauce, and “extreme Irish accent”. The negotiation ended up being pretty funny, especially since we asked AI to optimize for “chaotic humor and memorable melodrama.”

Everyone ended up getting one of their top choices! The results were also pretty consistent across ten tests. This is an advantage of AI agent negotiation: it can get awkward when we humans do it, especially over multiple rounds. But as long as our bots know what we want, they can figure out the best logical distribution even if it takes a little while.

We generated a multi-turn semi-formulaic conversation within ChatGPT, and are working on more complex group conversation patterns going forward. Then we generated audio using ElevenLabs text-to-speech, with default voices for our friends, and our own cloned voices for MishaBot and AvaBot. DALL-E generated the video’s themed watercolor visuals for each round (above). Placing the 🤖 emoji over our avatar faces had to be done manually: we thought Code Interpreter might be able to detect faces and place the overlay, but that didn’t work. However, cutting circles out of avatars and resizing via the same method worked great.

Later on, we’d like interactive AI agents to engage during the cooking process itself. It would be cool to input mid-preparation dish visuals and get live help on what to do next. Like the time I had to Facetime a Southern mom to help rescue a gravy attempt going haywire … but on demand and for every cuisine. Or maybe a Gordon Ramsay version that hurls themed insults at you along the way … “you Yokohama Burro!”

Results

We wanted to learn and improve with each AI dinner party, so collected ratings and feedback after each one. There have been ups and downs! 🇹🇭🇮🇹 Thaitalian was variable in quality by dish. 🇯🇵🇲🇽 Japaxican went surprisingly really well. 🇬🇷🇰🇭 Greekbodian was us getting cocky and stretching too far before our recipe quality reflection systems were good enough. 🦃🤝 Friendsgiving was a reasonably solid alternative Tgiving meal. Written feedback was also AI-synthesized; takeaways are with each dinner below.

Here are short descriptions for each of our first four participatory AI dinner parties: how we prepared, what we learned from feedback, and how the system has been evolving. Based on the ratings, each dish was categorized into 🏆 winner, 🆗 mid, or 🔻 nope. For each dinner you’ll also see contrasting images: AI (generated) vs IRL (cooked).

01–230624: 🇹🇭🇮🇹 Thaitalian

As our first dinner, this one was all experimentation. Some unexpected winners emerged, and we were excited that it worked at all. Learnings included:

• Improve the timing and sequencing of the event, including earlier cocktails and better cooking vs eating schedule.
• Improve the timing and sequencing of the event, including earlier cocktails and better cooking vs eating schedule.
• Enhance the menu variety and accommodate dietary preferences more effectively, avoiding repetitive ingredients.

02–230819: 🇯🇵🇲🇽 Japaxican

There was a notable improvement in quality, most likely from prompting the model to think more like a restaurant. We also tried a themed icebreaker at the start, which was OK but could be more interesting. Learnings included:

• Streamline the cooking process to allow for more simultaneous cooking activities.
• Improve the speed and efficiency of cocktail preparation to enhance the group experience.
• Increase flexibility and reduce the rigidity of the icebreaker format.

03–230901: 🇬🇷🇰🇭 Greekbodian

This one was a stinker. The chefs were amazing, but are not miracle workers, and we prematurely thought we’d nailed a system for any cuisine fusion to work. Turned out that we needed better prompting guardrails and output reflection. Learnings included:

• Improve the organization of cooking resources and space to reduce chaos and stress.
• Enhance the clarity and authenticity of the themed cuisine to ensure distinct flavors and cultural representation.
• Improve clarity and accuracy of preparation steps and time estimates in recipes.

04–231105: 🦃🤝 Friendsgiving

Twists on traditional dishes and incorporating our negotiation agents added a fun element, and again unexpected gems emerged. Quality reflections kept things on the rails, but a lack of restaurant-level taste emphasis kept most dishes as mids. Learnings included:

• Improve kitchen space management to avoid overcrowding and ensure a comfortable cooking environment.
• Enhance the distinctiveness and uniqueness of the desserts.
• Optimize the timing and distribution of voice interactions throughout the service process.

Lessons

Using AI prompting well is more complex than asking a single question. Agent reasoning and planning will get there at some point, but not yet. In the meantime, to do more ambitious things with less uncertainty, here are some of the evolving tactics we’ve learned to turn fun experiments into reliable infrastructure:

• Break complex goals into smaller chunks. At this point, there’s only so much “smart” per response. Feed the process in bites, not mouthfuls.
• With next token prediction, AI thinks off the top of its head, so try to get the answer last. Reasoning first is better than justification later.
• Help AI think like a human by having it describe a recipe’s characteristics before generating: sensory experience, visual flair, realistic logistics, etc.
• Detailing output format gives more predictable results. Asking for markdown, and giving few-shot examples, makes responses better.
• Try things several times. You’re the boss and can keep asking for options or reflections until you get one you like. Even better to give feedback along the way.

[Microsoft Design, Eric Koziol, Akash Kesrwani, Dan Shipper, Linus Lee]

Next

AI dinner parties have been a great first step into “AI for fun.” We believe that AI can help make us more human. Personalized AI creativity can help to produce unique experiences which would have previously taken professional-level effort. So we’re going to keep pushing ahead. Since these first four dinners, we have thrown (and will write about soon):

• 05–240406: 🎂⚛️ Elemental Personalities bday: Everyone comes dressed as the atom that AI matches them as most similar to.
• 06–240421: ✡️🫓 Passover Seder: Bots negotiate dishes as picture avatars, and we get traditional / alternative / personalized recipes.
• 07–240505: 🌱📽 Date night: Small dinner to use what was overgrowing in our Gardyn, with recipe generation flow built out further.
• 08–240608: 🧇💭 Saturday morning memorytasting: Hot waffles served in PJs, with custom nut butters & milks, based on weekend memories.

We’re continuing to develop and experiment putting on new party formats, including AI-based 🧺🌞 Picnic games, 🤖💞 Speed dating, 🎄🎉 HolidAI partAI, and more. And will keep trying new things at AI/ML Test Kitchen in Upper West Side, NYC.

As we keep exploring unique intersections of physical + digital experiences to celebrate humanity + AI, you can follow along! We’ll share experiments, learnings, new formats, and events through our new company Partylab.ai.

💻 Website • 📸 Instagram • 🐦 X/Twitter • 💼 LinkedIn • 📚 Substack • ⚫ Medium • 📺 YouTube • 🕰 TikTok

If you’re interested in magical new AI capabilities, I will also cross-post these plus more about creative personal AI use cases at 🧰 HowIUse.AI.

To hear about events we’ll start selling tickets for, follow us on any of the channels above. At some point we may also productize the systems we build to make the events happen. And if you want to hire us to conduct a guaranteed-unique AI party experience for your friends or company, let us know at partylab.ai/contact!

01001100 01101111 01110110 01100101,

AI & ML