Having ChatGPT Build Lightweight Stateless Streaming Microservices For Me

pulsar-chatgptgenerated-functions

https://www.pulsarfunction.dev/

Now, I have included links to human written functions including those that I have written at the end for you to compare. None of the generated ones compare to those written by a committer and expert like Author David K, but some of these are good starters. When you phrase the question hinting what you want it gets you a lot further. I also found that running them a few times in a row can you give you better results. If things freeze or not look good, reloading the entire chat browser page has helped. A clean slate for the chat memory is good.

Functions Generated by ChatGPT for Apache Pulsar

Prompts I entered

1. Can you write me an Apache Pulsar function in Java to translate all text from Spanish to English?
2. Can you write an Apache Pulsar function in Java that takes a byte array and converts to a JSON schema that also adds a new field that converts IP address to location?
3. Can you write an Apache Pulsar function in Python that takes a JSON message and converts to a JSON message utilizing a JSON Schema and does a lookup against a Scylla database for one value?
4. Can you write an Apache Pulsar function in Java that takes a JSON message and converts to a JSON message utilizing a JSON Schema, lookups up the current value of a stock symbole and routes the results to a new Pulsar topic based on the name of the stock.
5. Write a Pulsar function in java that looks up AQI data via zip code sent into the function as a String and outputs a full json record with a json schema. Implement the REST AQI lookup.
6. Can you write an Apache Pulsar function in Java that takes a JSON message and converts to a JSON message utilizing a JSON Schema and does a lookup against a Scylla database for one value?
7. Can you write a Java Pulsar Function that handles Oracle CDC records?
8. Can you write an Apache Pulsar Function in Java that does a word count on a string and returns a JSON record with a JSONSchema to a new pulsar topic.
9. Can you write an Apache Pulsar function in Java that does a persistent sum of all the integers it receives?
10. Can you write an Apache Pulsar function in Java that formats phone numbers?
11. Can you write a Pulsar Function in Java that saves JSON data to SnowFlake?
12. Write an Apache Pulsar Java function to implement Sensor Quantiles calculations on engine values that will be send as part of a ProtoBuf record and return the results as a JSON record with an AVRO Schema.
13. Write a Pulsar Function in Java that converts GTFS traffic data to JSON.
14. Write a Pulsar Function in java that parses standard log format inputs into JSON
15. Create a Pulsar Function in Java that reads Coinbase data via REST API
16. Write a Java Pulsar function that stores JSON data with a JSONSchema to Delta Lake
17. Write a Java Pulsar function that stores Faker generated JSON data with a JSONSchema to Apache Iceberg on S3
18. Write a Java Pulsar function that stores Google Ad JSON data with a JSONSchema to Apache Hudi on a dockerized Minio and sends a status to a new Pulsar topic
19. Write a Java Pulsar Function that sends JSON messages to JMS
20. Create a Pulsar Function in Java that adds github event stream data to the output
21. Build a Pulsar Function in Java that is a rules engine
22. Build a Pulsar Function in Java that is a rules engine with Drools
23. Write a Java Pulsar Function that Reads osquery readings from a server
24. Write a Java Pulsar Function that runs osquery readings and sends them as json to pulsar
25. Write me a Java application on a Raspberry Pi the reads sensor data, builds a JSON object and sends it to a Pulsar topic with a JSON schema.
26. Write a Apache Pulsar Function in Java that ingests Discord messages via Discord API and send the data as JSON to a Pulsar topic
27. Write a Apache Pulsar Function in Java that ingests Slack messages via Slack API and send the data as JSON to a Pulsar topic
28. Write a Apache Pulsar Function in Java that ingests Twitter messages via Twitter API and send the data as JSON to a Pulsar topic
29. Write a Apache Pulsar Function in Java that ingests Github event messages via Github Event API and send the data as JSON to a Pulsar topic
30. Write a Java application that consumes from a Pulsar topic and writes to Apache Pinot realtime tables.
31. Write an Apache Pulsar Function in Java that takes JSON Schema as input and write to Apache Pinot realtime tables.
32. Write an Apache Pulsar Function in Java that reads from Apache Pinot and writes to Apache Pulsar topics as Schema JSON.
33. Write an Apache Pulsar Function in Java that reads Clickhouse data and writes to Apache Pulsar topics as Schema JSON.
34. Write an Apache Pulsar application in Spring Boot 3 that reads Facebook messages and runs a Deep Java Library BertQaInference on the data which it sends as JSON with a Schema to Pulsar topics.
35. Create an Apache Pulsar function in Java in the style of reactive programming that writes 1,000 messages batches to MinIO object storage directories
36. Connect Apache Pulsar Functions to Apache NiFi asynchronously in a way that will scale to 1,000,000 messages per second.
37. Create a Spring Boot application that consumes Meetup events as JSON and sends them to Pulsar topics with JSON schemas
38. Create an Apache Pulsar Function in Python that utilizes Facebooks Prophet time series library to predict against a window of event data from a Pulsar topic.
39. Could you do the same thing in Java?
40. Create a Web Crawler in Micronaut framework that sends web data to Apache Pulsar topics
41. Create an Apache Pulsar Function in Java that ingests live web clickstream data and publishes it to Apache pulsar topics.
42. Create an Apache Pulsar Function in Java that utilizes machine learning to route based smart balancing
43. Implement all code in my github https://github.com/tspannhw?tab=repositories as Pulsar Functions in Java (No internet access)
44. Write the perfect basis for any Apache Pulsar Function in Java
45. Write an Apache Pulsar Function in Java that converts to deduplicate records
46. Write an Apache Pulsar Function in Java that runs md5 on incoming JSON records from Apache Pulsar topics
47. Write an Apache Pulsar Function in Java that runs levenshtein distance between two latitude and longitude points on incoming Apache Pulsar topics
48. Write an Apache Pulsar Function in Java that runs url_encode on incoming Apache Pulsar topics
49. Write an Apache Pulsar Function in Java that runs a Bloom filter on incoming Apache Pulsar topics
50. Write an Apache Pulsar Function in Java that adds 7 days to a date on incoming Apache Pulsar topics
51. Write an Apache Pulsar Function in Java that adds implements Raft consensus utilizing Apache Ratis on incoming Apache Pulsar topics
52. Write an Apache Pulsar Function in Java that adds the current timestamp in UNIX format to an incoming JSON file with an updated schema
53. Write an Apache Pulsar Function in Java that routes incoming Pulsar topic JSON events with schemas to randomized output topics
54. Write an Apache Pulsar Function in Java that adds a UUID as a key and primary key to an incoming Apache Pulsar topic before publishing it to a new topic with a Protobuf schema
55. Write an Apache Pulsar Function in Java that adds a UUID as a key and primary key to an incoming Apache Pulsar topic before publishing it to a new topic with a JSON schema

Preparing for Human-AI Convergence, Cats and Christmas

Code

For the Java and Python code developed, fork my github repo.

https://github.com/tspannhw/pulsar-chatgptgenerated-functions

Let’s look at some of the generated functions

Some like AQILookup gave us some code to retrieve the data from the source with comments.

private static final String AQI_API_ENDPOINT = “https://api.airvisual.com/v2/nearest_city";

@Override public String process(String zipCode, Context context)

{ // Make a request to the AQI API using the provided zip code to get the AQI data String url = AQI_API_ENDPOINT + “?zip=” + zipCode;

String response = makeRequest(url);

// Parse the response and return it as a JSON string return parseResponse(response);

}

Some gave the entire code needed for simple examples like add 7.

public class AddSevenDaysFunction implements PulsarFunction<String, String> {
  @Override
  public String process(PulsarMessage<String> input) {
    // Parse the input string into a LocalDate object
    LocalDate date = LocalDate.parse(input.getValue(), DateTimeFormatter.ISO_DATE);

    // Add seven days to the date
    LocalDate newDate = date.plusDays(7);

    // Return the modified date as a string in the ISO-8601 format
    return newDate.format(DateTimeFormatter.ISO_DATE);
  }
}

Some like the Discord parser gave me some proper field names.

Map<String, Object> message = GSON.fromJson(messageJson, Map.class);

// Extract the relevant fields from the message
        String channelId = (String) message.get("channel_id");
        String userId = (String) message.get("user_id");
        String userName = (String) message.get("user_name");
        String content = (String) message.get("content");

   // Build a JSON object with the extracted data
        Map<String, Object> discordMessage = new HashMap<>();
        discordMessage.put("channel_id", channelId);
        discordMessage.put("user_id", userId);
      discordMessage.put("user_name", userName);
        discordMessage.put("content", content);

        // Send the message to the "discord-messages" topic on Pulsar
        context.newOutputMessage(
            "persistent://public/default/discord-messages",
            Schema.JSON(Map.class)
        ).value(discordMessage).sendAsync();

The Coinbase example looks like it will work and I will try to compile and deploy this one.

public class CoinbaseFunction implements Function<String, String> {

  private static final String API_KEY = "your-api-key";

  @Override
  public String process(String input, Context context) {
    try {
      // Create a new URL object for the Coinbase API endpoint
      URL url = new URL("https://api.coinbase.com/v2/prices/BTC-USD/spot");

      // Open a connection to the Coinbase API
      HttpURLConnection con = (HttpURLConnection) url.openConnection();

      // Set the request method and add the API key to the request headers
      con.setRequestMethod("GET");
      con.setRequestProperty("CB-ACCESS-KEY", API_KEY);

      // Read the response from the Coinbase API
      BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
      String inputLine;
      StringBuffer response = new StringBuffer();
      while ((inputLine = in.readLine()) != null) {
        response.append(inputLine);
      }
      in.close();

      // Return the response as a string
      return response.toString();
    } catch (Exception e) {
      // Handle any errors
      e.printStackTrace();
      return null;
    }
  }
}

Some like Clickstream gave the most bare bones empty shell.

    @Override
    public Void process(String input, Context context) {
        // Parse the input string to extract the web clickstream data
        // ...

        // Publish the clickstream data to the appropriate Apache Pulsar topic
        context.newOutputMessage("web-clickstream-topic").value(clickstreamData).send();

        return null;
    }

Sometimes it would give me 80% of the code and then stop functioning before finalizing it. Often including all the imports you needed.

pulsar-chatgptgenerated-functions/DroolsEngineFunction.java at main ·…

On a second run it generated different code without imports.

pulsar-chatgptgenerated-functions/DroolsEngineFunction2.java at main ·…

Sometimes it would give you a good starter on how to ingest the data.

pulsar-chatgptgenerated-functions/GtfsToJsonFunction.java at main ·…

It does not seem to understand how Schemas and JavaBeans work, but kind of gives you somethings you can start with.

context.newOutputMessage("new-topic", Schema.JSON(String.class))

Some look really decent and may be the entire code. I am going to have to do a lot of testing.

pulsar-chatgptgenerated-functions/IcebergJsonFunction.java at main ·…

Now it is great to give me the function class, but I need to generate maven projects and add this code, setup pom file, build, generate deployment scripts and more.

Ideas for more useful Pulsar functions

• https://github.com/klout/brickhouse
• https://github.com/brndnmtthws/facebook-hive-udfs
• https://docs.oracle.com/cd/B28359_01/server.111/b28286/functions001.htm#SQLRF51171

Human Expert Written Functions

• https://github.com/david-streamlio/pulsar-in-action
• https://github.com/datastax/astra-streaming-examples
• https://github.com/david-streamlio/GottaEat
• https://github.com/jerrypeng/pulsar-functions-demos

Pulsar Sources/Sinks

• https://github.com/streamnative/pulsar-io-lakehouse

Example Human Written Functions

• https://github.com/tspannhw/pulsar-transit-function
• https://github.com/tspannhw/pulsar-energy-function
• https://github.com/tspannhw/pulsar-bertqa-function
• https://github.com/tspannhw/pulsar-chat-function
• https://github.com/tspannhw/pulsar-sentiment-function
• https://github.com/tspannhw/pulsar-pychat-function
• https://github.com/tspannhw/pulsar-airquality-function
• https://github.com/tspannhw/pulsar-adsb-function
• https://github.com/tspannhw/pulsar-weather-function
• https://github.com/tomaszmichalak/pulsar-sandbox
• https://github.com/yabinmeng/pulsar_function_example
• https://github.com/labuladong/play-with-pulsar
• https://github.com/Lanayx/fsharp-ukraine-2020/tree/master/src/PulsarFunctions

References

• https://www.pulsarfunction.dev/
• https://pulsar.apache.org/docs/2.10.x/functions-overview/
• https://docs.streamnative.io/platform/v1.1.0/concepts/pulsar-function-concepts
• https://github.com/ColinEberhardt/awesome-public-streaming-datasets