Automated Cognitive Meeting Minutes

Whether we’re attending general staff meetings or getting together with classmates to work on a group project, no one ever wants to be the note taker. You can’t pay as much attention to the meeting since you have to concentrate on taking notes, and when meetings get long, it’s often difficult to maintain the attention required to take diligent notes.

Do your hands cramp up at the thought of recording meeting minutes? Do you question what information you should record? Generally, minutes of meeting have been done manually by writing on paper or directly on a laptop while chairing a meeting or either type them up after the meeting.

Is it difficult to participate in the discussions at the same time? I don’t think you can make meeting notes and participate in discussions simultaneously. And it’s a huge waste of time in making meeting minutes either during meeting or after it. And people get frustrated by managing meeting minutes. Surely there has to be a better way?

That’s why our team (Akhil, Kashish and Rishab) decided to make an AI-based android app, as a final year project of our engineering at MIET Jammu, which can help you to take meeting notes.

Our AI-Based android app listens the meeting conversations, transcribes and then gives you the meeting minutes draft with decisions and to-dos.

But how well do it really work? It might not be that perfect but it takes just an effort to edit than it would be to start from scratch.

It categorizes the items while keeping an eye on the agenda and the elapsed time. It automatically assigns tasks to owners and deadlines. It summarizes the meeting and extract decisions, tasks and highlighted areas from the conversations.

The app also interacts with participants after a meeting is over to ask the attendees if anything is unclear. You can edit the text as you want, which you need to do for all meetings. When you are happy with your meeting notes, you can share it via email to the participants of meeting. How’s that for efficient?

It helps you stay focused on your meetings and makes sure that they are relevant to the work you are doing. Automatically transcribes meetings so you can focus on what’s being said, not what to write. It encourage speakers to not talk over one another or to interrupt another speaker.

We expect the app to get better over time. The AI in meeting minutes app will learn your business jargon and get better at adapting to your situation.

# Features:

1. Real time transcripts generation using Speech to text using IBM Watson API

3. Automatic speaker segregation

3. Sentiment analysis of speakers involved in the meeting

4. Make notes and find important keywords in meetings

5. Allocate and Assign tasks to the members of the meetings

6. Store all the meetings details in Firebase

7. Sharing of meeting minutes via email.

Architecture

How it works and Technology behind it:

It uses Watson Speech-To-Text technology as well as other cognitive applications to turn your meeting into structured meeting minutes.

During recording of meeting, it makes a request to the server to transcribe the speech into text using the IBM Watson Speech to Text API. It will transcript the meeting at real time and also does the speaker diarization like “who spoke when” and will flag the content accordingly.

But at the core of its, are Natural Language Processing (NLP) algorithms;

• The final transcript is first preprocessed, involving tokenization, stemming, and automated punctuation. Automated punctuation is accomplished using supervised machine learning, entailing a recurrent neural network model trained on over 40 million words.
• The Transcript Analyzer then integrates with the IBM Watson Natural Language Understanding API to detect keywords, topics, and concepts in order to determine the overarching theme of a meeting. We analyze the connections between these three categories to determine the most important topics discussed during the meeting which is later added to the email summary.
• We also isolate dates and times to be added to the calendar. When a date or time is isolated, the NLP algorithms search surrounding text to determine an appropriate title as well as key points. Even keywords such as “today”, “tomorrow”, and “noon” will be identified and appropriately extracted.
• Action items are isolated by searching for keywords which are useful such as, ‘decisions taken’, ‘issues raised’, ‘suggestions’, ‘next schedule’,etc. in the transcript and these action items are processed by performing POS tagging, facilitated by a trained machine-learning module, ultimately being appended to the final meeting summary of the most important points discussed.

Meeting minutes created by placing action items, to-do’s, dates, time, important points extracted by NLP algorithms into structured meeting minutes. After meeting is over, it can be shared to all participants via email.

APPROACH FOLLOWED BY OUR TEAM

It started with our minor project where we were able to execute the summarization part of our app.

Akhil and Me(Kashish) worked on the ML part of the app like speech to text conversion, summarization, keyword extraction, sentiment analysis and the automatic mail sending code and implementations were performed by us together. Rishab on the other part was working on the development of app and integration of ML code into it.

There were many problems faced by us during the development of such app. Though it looks easy that we only have to use some APIs integrate them and run our app. But this itself was the toughest part. Usage of API was not easy for. Sometimes we faced challenges with our own code and the reliability on ML part was not upto the mark. API usage was somehow learnt by us referring many videos but still sometime hope was lost when the results were poor. Github played a great role for reference and allowed us to refer some codes to corrrect our errors.

After months of efforts, our app started giving some expected results with some drawbacks. It can work on older version of android only and we are working on updation of libraries. On the other part, it works slowly and needs a large ammount data for summarization due to which it needs a good internet speed. And the work is still on.

It is now a major project for us and we are hoping to carry on with it till it will give us some fruitful results.

Let’s face it: There are better things in life than writing meeting minutes!