Extract text from images in 3 simple steps

Unleash the power of Microsoft Azure Computer vision using Python

OCR optical character recognition or to say in layman terms extracting text from images has been one of the most remarkable features available today. Gone are the days when organizations employed thousands of people just to type the text from images! Today, it is a job of a few people who can process thousands of images in just few minutes. Extracting text from images has helped organizations to massively improve customer service, increase storage spaces and secure the sensitive data in a compliant manner.

What will we Discuss?

We would learn how to extract text from images using the power of Microsoft Azure computer vision cognitive services.

1. We would use a sample image stored in Azure container Blob storage (https://jayantml1356189034.blob.core.windows.net/jayantcontainer/0001.jpg)
2. Process the given image to extract 100+ words.
3. Store the extracted result in a local MS Excel file.

Resources Required

• Python instance (i.e. Spyder IDE)
• Microsoft Azure Subscription (to run computer vision cognitive services and storage service to store images in Blob)

Step 1: Create Azure Computer Vision

1.1 Login to the Azure Portal: https://portal.azure.com/#home, search for “ Computer Vision ”

1.2 Create computer vision service by selecting subscription, creating a resource group (just a container to bind the resources), location and pricing tier. A free web container allows 5,000 transactions free per month. After clicking “Review + Create” , Azure may take a couple of minutes to create the resource.

1.3 Once the computer vision service has been created, navigate to “Keys and Endpoint” and copy the keys and endpoint details probably in a notepad.

⚠️ Please keep a note that keys and endpoint should not be disclosed to unauthorized people as they may impact your azure consumption cost. Regenerate keys if you have accidently disclosed the same.

Now, you are done with the Azure Portal portion now and can navigate to Python (Spyder IDE).

Step 2: Install AZURE packages

Installation: Open Python instance (i.e. Spyder 🐍 ) and issue below commands to install the required azure packages.

pip install — upgrade azure-cognitiveservices-vision-computervision
pip install pillow
pip install azure-storage-blob

Step 3: Python code magic

Use the below script and replace (a) Subscription key (b) Endpoint and Execute! You can try executing the script with other images by replacing the field ‘remote_image_handw_text_url’. The script has been updated with self-explanatory comments. Feel free to ask any further questions in the comments section.

This is how your output looks like. You can download the Excel here.

Conclusion

We learned 📘 how to extract text from image and get the output in MS Excel for further analysis.. You could use other images and customize the code to see what suits your use case best! 👍

Have questions?? Please drop it in the comments !

References

[1] https://docs.microsoft.com/en-in/azure/cognitive-services/computer-vision/quickstarts-sdk/client-library?tabs=visual-studio&pivots=programming-language-python

Checkmate the OCR Challenge: Image to Text Extraction was originally published in Geek Culture on Medium, where people are continuing the conversation by highlighting and responding to this story.

Key Phrase Extraction and Visualization: Python and Microsoft Power BI

Discover insights in unstructured text

Implementing RAKE algorithm in Python and Power BI integration

We live in an age where data is the new currency! This makes the Big tech giants the richest companies in the world. The best investment for the next few decades will be the investment in data. So, what do these companies do with this data? How can anyone handle pieces of textual and unstructured data from Facebook posts, Twitter or Linkedin? To a layman, scanning or sampling might sound like a good idea, however, data scientists know the risks of sampling and the pain of scanning text by text, row by row, and word by word 😬. This is where data experts use “Key-phrase Extraction”.

Key-phrase Extraction is the skill to evaluate unstructured text and returning a list of key phrases. For example, given input text “The food was delicious and there were wonderful staff”, the service returns the main talking points: “food” and “wonderful staff”.

What will we Discuss?

In this story, we will extract key-phrases using RAKE algorithm in Python on a sample set of data and then and visualize in Microsoft Power BI.

Here is the link for the sample data that we will use: Sample Data

What is RAKE?

RAKE is short for Rapid Automatic Keyword Extraction algorithm, it is a domain-independent keyword extraction algorithm that tries to determine key phrases in a body of text by analyzing the frequency of word appearance and its co-occurrence with other words in the text.

Resources Required

• Python instance (i.e. Spyder)
• Microsoft Power BI Desktop (Pro License)
• (OPTIONAL) Microsoft Azure Subscription (Free Trial or Paid) to correlate key-phrases together with sentiments.

Are you ready?? Here we go 🏄

Step 1: Install RAKE package and store stop wordlist

1.1 Installation: Open Python instance (i.e. Spyder 🐍 ) and issue below command to install the rake package.

!pip install python-rake==1.4.4

1.2 Create stop wordlist: Stop words are the words that generally do not help in text analysis and are typically dropped within all the informational systems and also not included in various text analyses as they are considered to be meaningless. Words that are considered to carry a meaning related to the text are described as the content bearing and are called content words. You can download the stopwords list here and customize the same as per your requirements. Save it at the desired location and copy the path for configuring the Python script.

Step 2: Open Power BI, Import Data & Configure Python script

2.1 Power BI Data Import: Open a new instance of Power BI desktop>> Import Data from Excel (Sample Data) >>Browse the sample data file >> Import data >> Calling “Run Python script” in Power Query Editor (Under Transform)

2.2 Prepare your Python Script: You can use the below Python script and customize the same by replacing the path for stopwords list in row 11.

Also, you can specify/restrict the # of key-phrases to be extracted by modifying the count in row 31 (i.e. replace [-1:] to [-5:] to get up to 5 key-phrases from 1 text input)

Once done with the customization, you can apply the script and expand the “Rake_Final_Output” dataset. You can Save and Close the Power Query Editor to apply the script. This is how your dataset looks like after new fields added for key-phrases and their scores.

Step 3: Power BI Integration and Visualization

Now comes the fun part that we all love, the visualizations! 💝

In order to visualize the key-phrases, I would recommend to use a Word Cloud ☁️ together with tables preferably with sentiment analysis 😃, so you can relate the key-phrases with positive, neutral and negative sentiments.

You can download a sample Power BI template which integrates Sentiment analysis as well as key-phrase extraction all packed together in a Power BI.

As you can see in the below example, we have “Top 10 Key phrases with negative sentiments” where phrases like “Slower Connections” and “restart 10 times” are directly correlated to negative sentiments 😢

Similarly, we have “Top 10 Key phrases with positive sentiments” where phrases like “explained neatly” and “great in depth knowledge” are directly correlated to positive sentiments 😃.

Conclusion

We learned 📘 how to apply RAKE algorithm to extract key-phrases and integrate the analysis in Microsoft Power BI to develop visualizations.

You could use other datasets and customize the code to see what suits your use case best! 👍

Came across a different approach for key-phrase extraction? Please drop it in the comments !

References

[1] https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/tutorials/tutorial-power-bi-key-phrases

[2] https://towardsdatascience.com/analyzing-and-visualizing-sentiments-from-unstructured-raw-data-c263ba96cc2c

[3] Data source: prepared manually by the Author

Key Phrase Extraction and Visualization : Python and Power BI was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

PMI AH-MC Micro-Credential

Hybrid is the new normal

Here is your chance to learn and realign to the latest project management standards. Did you know that the new PMP (Project management professional) includes agile and hybrid methodologies? 📚

I think being among the first in the world 🌎 to obtain the Micro-Credential: Agile Hybrid Project Pro (AH-MC) during Beta release from the Project Management Institute, I have another great opportunity to share my knowledge and experience.

So What is PMI Agile Hybrid Project Pro?

According to the PMI, this new credential demonstrates knowledge of agile and hybrid project management concepts, tasks and techniques that are applicable in virtually any industry. You can find more information about the micro-credential at the following link: https://www.pmi.org/landing/agile-hybrid-micro-credential

Agile Hybrid Project Pro is a micro-credential product that includes an eLearning course and an assessment. It assesses Agile & Hybrid project management knowledge & skills through a short, focused examination. The eLearning course will help you in preparing for the assessment.

The credential will be soft launched in Apr 2021.😃

You may love the fact that this micro credential badge never expires! but for me it was venturing into the hybrid methodology trying to corelate how agile and waterfall can work together.

Who is eligible?

1. It is ideal for traditional project managers who are beginning to venture into the Agile or Hybrid space.

2. It helps current Project Management Professional (PMP)® certification holders align to the new PMP® standards, which includes Agile and Hybrid methodology. Further, you get 13 PDU’s to help you to renew your certification.

Exam Format

You have 75 minutes to answer 60 questions.

So we take the eLearning course, take assessment and earn the credential. Is it correct?

To be honest, I think that the e-learning course is not sufficient. Remember, to pass any PMI certification/credential, you should have aligned your thinking with the methodologies and concepts which in this case is Agile as well as Hybrid.

This is very tricky with this new credential as you fall into the trap of confusion between traditional waterfall and agile approach. For example, in a hybrid environment, in terms of artifacts what will be useful in a given scenario? A risk register or Sprint Backlog? Similarly, Who will do xyz activity? Project manager or Team?

Ok, this seems to be very difficult?

Well, the answer is No. If you understand the context then you will be able to fit into the scenario in a hybrid environment and pick the correct option.

I am sharing few sure shot tips that can help:

1. Read the first 4–5 words very carefully to understand the project setting. i.e. During an agile project VS During a project.
2. Understand that in agile environment, project manager is comparable to Scrum master and acts as a servant leader and Servant leaders facilitate.
3. Go through PMBOK and Agile practice guide.
4. Remember the taxonomy: predictive=Waterfall= Traditional, whereas Agile=iterative=sprints
5. Remember that as per Agile Practice guide, ideally the transition from waterfall to hybrid should be gradual. This would require learning and alignment between teams.
6. Understand the conflict resolution techniques i.e. Collaborating, Avoiding
7. Daily stand ups are important.
8. Read and understand everything in the official Scrum Guide
9. Product owner takes scoping decisions.
10. Don’t forget to take a breathe during the exam. Oxygen is vital for the functioning of our brain. ☺️

Conclusion

Last but not the least, Believe in yourself and don’t be afraid of the outcomes.

I would like to quote one of my inspirations A. P. J. Abdul Kalam (Teacher as he liked to be called instead of Indian aerospace scientist and former president of India) that Fail stands for “First Attempt in Learning”

The beautiful thing about learning is that nobody can take it away from you.

Still have questions in your mind?? Feel free to drop a message below.

References

[1] https://www.pmi.org/landing/agile-hybrid-micro-credential

[2] https://www.pmi.org/

[3] “PMI,” the PMI logo, “PMP,” “Project Management Institute” and “Pulse of the Profession” are registered marks of Project Management Institute, Inc.

Using Microsoft Azure Text Analytics and Power BI Integration

The Sentiment Analysis

Microsoft Azure Cognitive services and Power BI Integration

The total amount of data created, captured, copied, and consumed in the world is forecasted to increase rapidly, reaching 74 zettabytes in 2021 (Source: Statista). Imagine how much of this 74 zettabytes data will be unstructured and untamed, leaving a huge void 💣in how data scientists around the world would analyze, model and consume this mammoth amount of data. For example, product reviews on ecommerce websites like Amazon or free speech on social media giants like Facebook and Twitter.

One of the key challenges with such with unstructured data is how to gauge public opinion, conduct nuanced market research, monitor brand and product reputation, and understand customer experiences. In layman terms, how to understand and classify emotions or for data scientists how to perform a sentiment analysis. A Sentiment Analysis is the process of determining whether a piece of writing is positive, negative or neutral.

What will we Discuss?

In this story, we will perform sentiment analysis on a sample set of data and use :

1. Microsoft Azure Text Analytics (For performing sentiment analysis)
2. Power BI (For integration and Visualization)

Here is the link for the sample data that we will use: Sample Data.

Resources Required

• Microsoft Azure Subscription (Free Trial or Paid)
• Microsoft Power BI Desktop (Pro License)

Are you ready?? Here we go 🏄

Step 1: Azure Text Analytics

Login to the Azure Portal: https://portal.azure.com/#home, search for “ Text Analytics ”

Create a Text Analytics service by selecting subscription, creating a resource group (just a container to bind the resources), location and pricing tier. A free web container allows 5,000 transactions free per month. After clicking “Review + Create” , Azure may take a couple of minutes to create the resource.

Once the text analytics resource has been created, navigate to “Keys and Endpoint” and copy the keys and endpoint details probably in a notepad.

⚠️ Please keep a note that keys and endpoint should not be disclosed to unauthorized people as they may impact your azure consumption cost. Regenerate keys if you have accidently disclosed the same.

Now, you are done with the Azure Portal portion now and can navigate to Power BI.

Step 2: Power BI Integration

Open a new instance of Power BI desktop>> Import Data from Excel>>Browse the sample data file >> Bingo! You got your dataset imported in Power BI inching closer to the world of visualization.

Use the APIKey and Endpoint link from Step 1 and replace the placeholders in below M query script, which basically helps to perform API calls to Azure. Another important parameter to know here is the language: ““en”” which can be tweaked to include more than 20 languages (Hindi, Chinese, German, French to name a few).

Now go ahead and select Get data>>Blank query (this will navigate to Power Query editor)>> Paste the M query code (containing replaced API Key and Endpoint link)

It will be a 🔦 good idea to rename the “Query1” to something meaningful like “Sentiment API”. Go ahead and edit it either by right clicking on left hand side pane or directly editing in Name field on right hand side pane.

Now, in the power query editor itself navigate to the dataset and under Add Column section>>select “Invoke Custom Function”>>Add column name=Sentiment, select function query from drop down and pass text as the column with feedback “Answer”>>Press OK.

This will invoke the Azure API and create a new column, Expand the column “Sentiment” and change the data types for 3 newly created columns (positive, neutral, negative) to “Whole Number”. While you can keep them in decimal format, I would recommend to change them to whole numbers as it helps to visualize them better. Now you can hit “Save & Close” and let the Power BI do its magic! 🎉

Before you finally enter visualizations, it will be useful to create a new calculated column which can be used to provide slicers for easy filtering. You can collate everything into one column using below DAX formula.

Overall Sentiment = IF(Responses[Sentiment.positive]=1,"Positive",IF(Responses[Sentiment.neutral]=1,"Neutral",IF(Responses[Sentiment.negative]=1,"Negative","NA")))

Step 3: Power BI Visualization

Now you are ready to visualize 📊 the information. I would recommend to use the following approach:

1. A horizontal slicer to filter sentiments by Positive, Negative and Neutral.
2. A bar chart to represent sentiment trend by week, month or year.
3. Some emojis to represent the feeling and emotions of the audience.
4. A table providing all the raw data fields to correlate and “Export to Excel”.
5. A disclaimer to specify the limitations of the AI model.

*Being a predictive AI model, this may not be 100% accurate representation of the sentiment but it does helps with actionable insights and quicker decision making providing indicative sentiments. #responsibleAI #transparency.

Last but not the least, you can also add conditional formatting to ensure color schema matches with sentiments.

Here is an example of sample Power BI template that you can use as a starting point. Sentiment Analytics Azure + Power BI Integration DEMO.pbix

Output

By slicing and dicing the filter you can see how effective Azure text analytics resource in performing sentiment analysis. For example, feedbacks like “good session” and “wonderful event” were correctly classified as positive 😃 and similarly feedbacks like “slower connections” and “lot of problems” were correctly classified as negative 😞.

Conclusion

We learned 📘 how to use Microsoft Azure Text Analytics for sentiment analysis and how to integrate the analysis in Microsoft Power BI to develop visualizations.

You could use other datasets and customize the code to see what suits your use case best!

Came across a different approach for sentiment analysis? Please drop it in the comments !

References

[1] https://www.statista.com/statistics/871513/worldwide-data-created/#:~:text=The%20total%20amount%20of%20data,ever-growing%20global%20data%20sphere.

[2] https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/tutorials/tutorial-power-bi-key-phrases

[3] Data source: prepared manually by the Author

Analyzing and Visualizing Sentiments from unstructured raw data was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

So, the first question, What is PMI? What is PMP? Well, PMI is Project Management Institute which is the world’s leading association for those who consider project, program or portfolio management their profession.

I started my journey with PMI in 2017 while I saw my ex-manager’s Linkedin profile with suffix called “PMP” :) Almost immediately, I started researching about the PMP (Project Management Professional) certification and learnt how valuable is the knowledge and credibility for my career. As I was handling a lot of projects, so I could literally relate to most of the concepts right away like Planning, Scope, Costs, quality etc.

Next step, I took PMI membership, submitted application, booked the exam, studied day and night, took some time off before exam, did 10 mock tests and 20 days after Bingo!!!! I became Jayant Kodwani PMP.

For many aspirants, PMP is just a magic wand which can help them earn higher salaries. I don’t believe it at all. There is a difference between “Acquiring knowledge” and “Applying Knowledge and gaining experience”. With PMP certification you get knowledge and tools which are invaluable (*for those who take it seriously) but ultimately it is your attitude and determination that can take you to heights. let me quote here

Knowledge unused is worthless; only knowledge applied and shared is power.

So, here is what I did, logged into wordpress and started my blog https://jayantkodwani.com/ and shared my experience and learnings. I used my expertise in Excel, VBA and created some of the unique tools in the field of project management. Feel free to access them below.

(Mastering PERT/BETA/3 Point Estimate and Six Sigma PMP Questions)

PMP EVM (Earned Value Management) terms and formula calculator with Mock Exam in MS Excel Spreadsheet

For me, PMP certification opened the gateway of ultimate happiness where on one side, I could not resist to learn more about project management (2 months later, I did PRINCE2 certification) and on the other side I had the joy of giving, sharing, mentoring and helping others to learn and acquire the knowledge.

With that note, I am closing this story and promise to share lot more in upcoming posts. Don’t forget to follow me on Medium :)

Feel free to connect to my network : https://www.linkedin.com/in/jayantkodwani/

and follow my blog at

https://www.jayantkodwani.com

Before you get that call for an interview, its your resume which can make or break a connection with the recruiters and hiring managers. A strong and impactful resume should have the power to communicate with the right set of comprehension and word selection. While many people nail this art with ease, others struggle with what to write, what to highlight and how to summarize and end up using what is known as “Buzzwords“. These buzzwords sounds great to read but actually do not add any value to your credibility. Here are the top 10 buzzwords that you should delete from your resume asap.

1. motivated/highly motivated
2. hard working
3. energetic
4. punctual
5. loyal
6. enthusiastic
7. team player
8. client-focused
9. a people person/leader
10. good communicator/ interpersonal skills

• So I cleaned up resume but now it looks deserted
• How will a recruiter assess my resume?
• What next?? Why did you suggest to even remove these?

The problem with buzzwords is that they are not backed up with your experience and achievements. They appear to be fake and often copy/pasted from other fellas, So, the answer is that you should add your actions, examples, evidences and how you demonstrated those qualities. This can be done when you use verbs like:

1. As a part of project team, I contributed by….
2. I developed…..
3. I initiated/took an initiative….a daily stand up…
4. I conducted….sessions for training/coaching…
5. Managed the….. project/transition…
6. I mentored…..
7. I ensured….
8. I transitioned…..
9. Provide evidence> I was recognized/awarded/nominated….
10. Provide evidence> I got certified by…..

Last but not the least, don’t miss I volunteered to add a cherry on the cake. To conclude, I would recommend you

• To check your resume and get rid of the buzzwords.
• Never ever let anybody else create/write your resume/ Do not copy stuff as is from your friends and colleagues. You might accidently add the buzzwords as well.
• Provide actions/outcomes/evidences/efforts.

I hope the above tips will help you in your job search journey especially during these tough pandemic times. Best of luck!

Feel free to connect to my network : https://www.linkedin.com/in/jayantkodwani/

and follow my blog at

https://www.jayantkodwani.com

Delete these words from your resume ASAP was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

Dear Readers,

Majority of tourists (domestic as well as Foreign) don’t know how to online book Tiger safari at Ranthambore through official means and end up booking safari at higher prices through Hotels and touts. Hotels/agents charge 40–70% higher than the genuine amount basis the season and availability. For example:- I have seen hotels/agents charging around 1200 INR per person (Indian) for a Canter Safari whereas the official charges for Safari booking as of 2018 is 582 INR per person (Indian) Canter. let me help you with more details.

The official website of Rajasthan Government for Wildlife Safari Booking is fmdss.forest.rajasthan.gov.in.

There is also a step-by-step PDF manual which will help you with the process and guidance for booking the tickets online.

Let me talk more about the things which usually nobody shares. Below is a copy of Booking confirmation Slip for a Canter Safari in Zone-3. This slip provides a lot of information for about Ranthambore Safari like the timings, boarding point, charges in Details and terms and conditions.

Safari Zones, Mode and Cost at Ranthambore National Park

Once you have registered on the portal, you will face questions like:

• Which safari zone is the best?
• What is the best mode of Safari?
• What would be the cost of Safari?

Below is the table with all the answers. However, talking about safari zones there is no guarantee of a Tiger sighting in any of the zones. It is all about sheer Luck and the probability. Though Zone 3 has much higher probability of Tiger sighting because of the water bodies, you may still end up without any Tiger Sighting. From my personal experience, I have visited Zone 3 and 6 and was unsuccessful of a Tiger Sighting, but I am optimistic and would surely try again.

Important Note for Foreign visitors

You might face challenges while trying to register on https://sso.rajasthan.gov.in/register?ru=fmdss as you don’t have Indian ID’s like Aadhaar and Facebook/Google/Twitter Logins don’t work most of the time. In case you face such issues, don’t be upset and feel free to contact me to facilitate the same. I can secure a booking for you from my Login ID. Why I am ready to help? because of the vision of this blog “ This Life is to help others “. How much will it cost? Nothing

The Boarding Point and time for Safari

You should reach the boarding point at-least 1 hour prior to the Safari Time. The boarding point is at Forest Safari Booking Office. Here is the exact location and address for the boarding point.

Boarding Point:- Forest Safari Booking Office, Shilpgram, Village Ramsinghpura, Ranthambhore Road, Sawai Madhopur

Below is the Google Map location for the boarding point.

https://jayantkodwani.com/how-to-book-a-tiger-safari-at-ranthambore-national-park/

I hope this article was helpful. Wishing you Good Luck for a Tiger Sighting at Ranthambore.

Feel free to connect to my network : https://www.linkedin.com/in/jayantkodwani/

and follow my blog at

https://www.jayantkodwani.com

Originally published at https://jayantkodwani.com on May 3, 2018.