Langflow Micro Tutorials — Batch Web Loader
Welcome back to our Langflow micro tutorials series! In this article, we’ll continue exploring simple Langflow examples and custom component design.
You can download the flow mentioned in this article to modify and understand the components used.
Today, we’ll focus on a web scraper that loads multiple websites and runs the same prompted LLM on each one of them. Hope you enjoy it!
Main Features
Batch Web Loader: This Custom Component leverages the functionality of langchain’s WebBaseLoader by incorporating it into a for loop. This allows users to process multiple web-based tasks sequentially.
JSON Saver: Parses JSON outputs and stores them into files using one of the keys as the file name.
Objective
The objective of this flow is to extract information from webpages in JSON format using the structure output parser.
The flow focuses on web scraping features such as name, birth date, and birth location from a Wikipedia page.
The webpage field in the prompt template remains empty to allow for recursive filling by the for-loop inside the custom component created.
In this example, the URLs field was filled with the following webpages in a Python list format:
The output files are then created by the second custom component: “Mary Louise Streep.json” and “Tom Cruise.json”, which were stored in the Langflow installation folder (since no absolute path was provided).
