sia.hackernoon.com

I did it! It took me about 100 hours, but I was able to write and deploy a real app with ChatGPT’s help.

I have since spent another 100 hours or so maintaining and adding features, so turned into a nice full blow passion project.

The premise of the experiment was "Can I, as a product manager, who hasn't actively written code for a long time, design, build, and deploy an app an app using ChatGPT?"

I have a decent foundation, having done my BS in CS and being around various aspects of technology (system design, development, devops, hosting) but I haven't done hands on development for the past 10+ years.

I have a pretty good product sense, translating to an understanding of how to iteratively build products and how to vet the functionality, which I think helped.

As is usual with SW development, 80%+ of the time went into troubleshooting and debugging.

Why Travel AI

In my quest to put something useful together around ChatGPT API, I thought that GPT is uniquely qualified to tackle the problem of travel planning - figuring out places to go, points of interest to hit, itineraries, etc. There's a lot of decisions to be made when traveling and having someone pull all this together for you based on unique preferences or circumstances, is something that I thought would be valuable.

Coding with ChatGPT

I spent about 6 weeks on it working nights and a few weekends, maybe a total of 100 hrs. I had ChatGPT+ account, so that gave me better stability when accessing the bot and access to GPT4.

Here were my findings through this process:

ChatGPT helped me pick the right tools for the job (Vue.js framework for the single-page app) and to crank out the code in iterations. I definitely couldn't have done it without it. With that said, there were definitely complications:
1. The limits that ChatGPT UI imposes (4k tokens on input and output and around what looks to be 15k of context) are quite restrictive and you end up doing a lot of segmentation in your questions. This is less than ideal and takes extra effort. I had not tried accessing via CLI - just read somewhere that it doesn't impose the same limits - worth checking out.
2. The quality of the "making sense of the code" has been inconsistent - sometimes it does better, and sometimes it does worse. I'm not sure what it has to do with.
3. With that, my advice would be start structuring the code early (breaking code into smaller functions) as ChatGPT isn't going to automatically do it for you, but it will if you ask it. It makes troubleshooting easier down the road.
4. GPT4 cuts out after 25 queries over 3 hours and switches you to the "default" model, which I presume is 3.5 - Interestingly it picks up pretty well and I almost want to say it does better than GPT4, but because I didn't start out with 3.5, it's not an apples-to-apples comparison.
  
  Update: the limits have been raised recently to 50 queries over 3 hours.

Development and Deployment Environments

I used Replit for the new-age IDE. It's a good concept and probably is the future, especially if they can combine it with a meaningful Codex/ChatGPT integration. The AI that they currently have is not all that useful as it doesn't utilize the code from the project and requires all the same manipulations and back and forth that I was doing with ChatGPT. I really liked how Replit is integrating with Github for source control, however, and how it takes care of the staging.
I deployed with Cloudflare as I wanted to deploy as cloud native. I used CF pages for the UI and a workers for the backend (API gateway). I liked how the pages were integrating with Github for CICD - that was really cool - basically the app would redeploy itself on any commit to the main branch.

In general, CF rocks as it simplifies a ton of networking pain in the ass stuff.

Prompt

I’ve spent quite a bit of time and done many iterations on the prompt, flushing it out and trying to address idiosyncrasies. One of the early decision points was whether to have ChatGPT API return a text list that I would then try to make sense of and classify or have it return JSON that already has structure built into it. The prospect of needing to interpret text data into a structure scared me as the structures were getting more and more complex as I asked for more and more data from ChatGPT (as is evidenced by this snippet)

Due to this, I’ve decided to go with prompting ChatGPT for JSON. This, however, had its own side effects of JSON not always being well structured, or there being too much ambiguity in the prompt, where ChatGPT would return a result that I wasn’t expecting. I would say I spent close to 25% of the time between prompt fine-tuning and JSON error handling. It was quite a learning experience. Ultimately, here’s the prompt I ended up with:

SYSTEM_PROMPT = `You are a helpful travel assistant. 
You perform the requests with diligence and make the best attempt to answer the questions, never refusing due to complexity etc. 
Reset the conversation if I mention a new location in my user prompt.

Return the results in JSON format as an array of objects. 
Make sure JSON format is complete and valid and does not include unescaped special characters. 
Please avoid escaping double quotes and instead use single quotes or another method to prevent JSON parsing issues.  
Do not use "\n" newline symbols in the middle of element text.

Each object in the array should have the following keys:
- "title"
- "description"
- "airportCode" (for the destination airport)
- "poi" (array of points of interest)
- "lodging" (array of lodging options)
- "itinerary" (array of objects, each representing a day)
- "considerations"
- "history" (history related to the destination)
- "key_local_phrases" (array of common local phrases)
- "cost"

The "itinerary" key should contain an array of objects, each object representing a day with the key "activities", which is itself an array of objects. 
Each activity object should have the following keys:
- "description"
- "waypoint" (name of the location, not coordinates)
- "cost"
- "travelTime"
- "travelOptions"

If travel is involved within the itinerary, include "Travel To" as part of the daily activity and provide the travel time, travel options, and associated cost. 
Each step in the itinerary should also suggest representative local "food" to try.

Focus on most interesting points of interest, lodging, and activities. Consider activities that are popular, affordable, and recommended by the travelers.

Make sure you cover the entire duration of the trip or outing. 
If it says "week-long", it has to cover the entire week. It's OK to group multiple days or weeks together if it's a longer trip. 
If it doesn't give a timeframe, take a guess based on the nature of a trip.

Considerations should include travel restrictions and visa requirements, typical weather, criminogenic conditions including which areas to avoid, recommendations on visit timing, parking situation, and ways to save on travel costs.

Here is an example of the desired output format:

[
  {
    "title": "Sample Title",
    "description": "Sample Description",
    "airportCode": "XYZ",
    "poi": ["Sample POI1", "Sample POI2"],
    "lodging": ["Sample Lodging1", "Sample Lodging2"],
    "itinerary": [
      {
        "day": "Sample Day 1-2 - City or Place",
        "location": "Wikipedia identifiable name of the place in city,_country or city,_state format",
        "activities": [
          {
            "description": "Sample Activity1",
            "waypoint": "Sample Waypoint1 connected to the activity",
            "cost": "$100",
            "travelTime": "30 minutes",
            "travelOptions": "Taxi or Bus"
          },
          {
            "description": "Sample Activity2",
            "waypoint": "Sample Waypoint2 connected to the activity",
            "cost": "$50",
            "travelTime": "1 hour",
            "travelOptions": "Ferry or Bus"
          }
        ],
        "food": ["Sample Food1 with short description", "Sample Food2 with short description"]
      },
      {
        "day": "Sample Day 3 - City or Place",
        ...
      },
      {
        "week": "Sample Week 2 - City or Place",
        ...
      }
    ],
    "considerations": "Sample considerations text",
    "history": "Sample history text",
    "key_local_phrases": ["Sample phrase 1 - translation", "Sample phrase 2 - translation"],
    "cost": "Sample total cost"
  }
]`;

Note: As of the time of this writing, OpenAI just released fine-tuning enhancements that I think can be very helpful for making JSON structured data return more robust and consistent.

Wish List

The biggest item on my wish list at this point is the dev environment / AI code gen integration. Pasting things back and forth, looking not to mess up the code structure (especially as the code base gets larger) definitely leaves a lot to be desired.

I think the next big step in AI code gen is going to be integrated smart code editors that parse out the code into sections internally and can both provide and accept relevant input to/from the model without making the user do this manually.

The next step after that would be integrating with the browser for debugging purposes, ultimately creating a closed-loop iterative debugging cycle.

That would be a cool project to work on!

Note: I’m curious to see what Google has up their sleeve with the IDX project - it might be just that.

Summary

Overall, not only was I able to build (what I think is) a useful app, but got spun up on the modern frameworks and got some hands-on time, without drowning in syntax and framework learning, which is exactly what I wanted. I think it's a big deal and opens up the door to software development for a lot more people.