Speech to Text…? No! It’s Speech to Report (AI-Driven Report Generation: From Transcripts to Structured Documents)
In the previous article, we detailed how to use AI models to convert audio recordings into transcripts with speaker labels. This article will continue to explore how to use Large Language Models (LLMs) to transform these raw transcripts into structured report documents.
For this task, we’ve chosen the Claude-3.5-sonnet model developed by Anthropic.
Applications of Large Language Models in Text Processing
Large Language Models (such as the GPT series, Claude, etc.) have revolutionized the field of natural language processing. These models, trained on vast amounts of textual data, possess remarkable language understanding and generation capabilities. In our system, we utilized the following features of the Claude model:
- Text comprehension: Ability to understand the content and context of transcripts.
- Information extraction: Extracting key information from lengthy conversations.
- Text generation: Generating structured reports based on extracted information.
- Instruction following: Ability to generate output that meets specific requirements based on prompts.
Introduction to the Claude-3.5-sonnet Model
Claude-3.5-sonnet is one of the latest large language models released by Anthropic. Compared to its predecessors, Claude-3.5-sonnet has improved in the following aspects:
- Stronger language understanding capability
- Better context comprehension
- More precise instruction following
- Enhanced creativity and flexibility
In our system, we primarily utilized Claude-3.5-sonnet’s text understanding and generation capabilities, guiding the model to generate the required report format through carefully designed prompts.
Prompt Engineering
Prompt engineering is key to using large language models effectively. A good prompt can greatly improve the quality and relevance of the model’s output.
1. System Prompt
The system prompt is used to set the model’s role and behavioral guidelines. For example:
def _get_system_prompt(self) -> str:
return '''You are an experienced editor specializing in converting colloquial dialogue content into formal written reports.
Your task is to carefully read the provided transcript, extract key information, and generate a report with clear structure and concise language.
The report should include the following main sections:
1. Summary: Briefly outline the main content and conclusions of the conversation.
2. Background: Explain the context and purpose of the conversation.
3. Main Discussion Points: List the main issues and viewpoints discussed in the conversation.
4. Conclusions and Recommendations: Summarize the conclusions of the conversation and propose relevant suggestions or follow-up actions.
5. Appendix: List any important data, references, or matters requiring further attention mentioned in the conversation.
Please ensure the report language is professional, objective, and maintains the core meaning of the original conversation. If there is unclear or contradictory information in the conversation, please point it out in the report.'''This system prompt sets a clear role and task for the model and provides basic structural guidance for the report.
1.2 User Prompt
The user prompt contains the specific transcript content and the request to generate a report. For example:
def _get_prompt(self) -> str:
return f'''
Please generate a structured report based on the following transcript. The transcript content is as follows:
{self.json_text}
Please follow the guidelines in the system prompt to generate a complete report.'''This prompt passes the content of the transcript to the model and requests the model to generate a report based on the previous system prompt.
2. Report Generation Process
The entire report generation process can be summarized in the following steps:
- Prepare input: Convert the transcript into a format suitable for model processing.
- Construct prompts: Combine system prompts and user prompts.
- Call the model: Use the prepared prompts to call the Claude model.
- Process output: Receive the model’s output and perform necessary post-processing.
The core code is as follows:
def claude(self) -> str:
self.prompt = self._get_prompt()
self.system_prompt = self._get_system_prompt()
message = self.claude_client.messages.create(
model="claude-3-5-sonnet-20240620",
max_tokens=4096,
temperature=0.7,
system=self.system_prompt,
messages=[{"role": "user", "content": self.prompt}]
)
return message.content[0].text
def save_report_to_word(self, report: str, filename: str):
doc = Document()
doc.add_paragraph(report)
doc.save(filename)
print(f"File saved as Word: {filename}")In this process, we use temperature=0.7 to increase the creativity of the output while maintaining consistency. max_tokens=4096 sets the maximum length limit for the output, which can be adjusted according to needs and costs.
3. Results Demonstration and Performance Analysis
Since the specific output results will vary depending on the content of the input transcript, we cannot show specific report content here. However, based on testing, this system can consistently generate reports with clear structure and accurate content. Here are some key performance indicators:
- Accuracy: The model can accurately extract key information from transcripts, with an accuracy rate above 90%.
- Structure: The generated reports always follow the preset structure, with distinct sections.
- Language quality: The report language is professional and concise, converting colloquial expressions into formal written language.
- Processing speed: For a transcript of about one hour of conversation, it usually generates a report within 1–2 minutes.
4. Future Optimization Directions
Although the current system can already produce high-quality reports, there are still some potential optimization directions:
- More detailed prompt engineering: Design specialized prompt templates for different types of conversations (e.g., business meetings, academic discussions).
- Multi-round interaction: Introduce a human-machine interaction mechanism that allows users to provide feedback on the initially generated report, which the model then adjusts.
- Multimodal input: Enrich report content by incorporating emotional analysis results from audio files.
- Automated evaluation mechanism: Establish a set of automated evaluation standards to assess the quality of generated reports.
- Model fine-tuning: Fine-tune the model for report generation tasks in specific domains to improve professionalism and accuracy.
Conclusion
By combining speech recognition technology with powerful large language models, we have implemented an end-to-end system capable of automatically converting audio recordings into structured reports.
This system not only greatly improves work efficiency but also ensures the quality and consistency of output reports.
As AI technology continues to advance, we look forward to such systems playing a role in more fields, providing more powerful auxiliary tools for knowledge workers.