EMPOWERING JOB SEEKERS IN THE MARKET WITH INFORMATION ABOVE AND BEYOND THE STATED JOB DESCRIPTION
Authors: Bhavana Prasad Kote, Maharsh Soni, Sachin Kumar Srinivasa Murthy, Rashmishree Veeraiah
Introduction:
The job market is characterized by dynamic shifts in the skills needed across all occupations. In order to adapt and progress in their career role at their selected company, job seekers must have precise knowledge of the shifting market needs that can assist them in continually improving and learning specialized tools. So, by analyzing thousands of job listings, we suggest a novel information systems technique that offers insights into the change in skill requirements at different granularities. Our system’s main goal is to provide job seekers with information that goes above and beyond the job description.
Motivation:
All aspects of the job market’s dynamics are constantly evolving. For starters, the demand for skills in a specific occupation has changed dramatically in recent years and is expected to change further as a result of technological improvements and a shift in organizations’ mindset toward sustainable growth goals. Secondly, today’s job candidates are motivated not only by total salary but also by things like great workplace culture, workplace diversity, and the impact of their work and contributions on society. To keep job seekers informed and assist them in staying relevant to the market’s needs, it is essential to follow the dynamic shifts in skill demand in the job market. In conclusion, our own needs as job seekers in the current market, as well as our comprehensive research on job boards and labor markets, serve as the foundation for our motivation to develop a data-driven solution.
In our project, we used different database techniques like AWS Redshift Query Editor V2, MySQL using python, and Neo4J + GraphXR to achieve solutions to our problem statements.
Project Flow:
Workflow 1:
- Preprocessed and cleaned raw CSV data using python’s PySpark and pandas libraries and stored them in S3. Data from S3 was then crawled to the Glue data catalog using crawlers.
- ETL was performed on glue and data was loaded to Amazon Redshift as a star schema with dimension and fact tables to perform analysis using Redshift Query Editor V2.
Workflow 2:
- Normalized, cleaned data in CSV files are imported into MySQL server using MySQL workbench.
- The MySQL server connection was established using Python/MySQL connector. Data Analysis was done using SQL queries in workbench and python.
Workflow 3:
- The Multi-valued attributes initially stored in a separate CSV file to adhere to First Normal Form were mapped to other essential attributes as required by the Neo4J data model to show relationships using the python pandas module.
- The new CSV file was imported into the Neo4J AuraDB workspace and the data model was constructed. Visual analytics is performed in the GraphXR Platform by connecting to the Graph database using the Neo4j API connection string.
Findings and Key Learnings
Some of the interesting observations through our analysis using RDBMS, Neo4j, and AWS Redshift include:
Data Analysis, Microsoft Excel, SQL, and communication skills are the most important skills for a data Analyst in the industry.
Those with bachelor’s degrees require twice the amount of experience to earn commensurate salaries as that of their master’s degree holder colleagues.
It is very important for job seekers to learn the difference in pay by location when deciding on their career in a location. Hence, we executed some of the queries to inform the job seekers by inputting their role, and firm name to understand the average max and min salaries across locations.
Conclusion:
Using Neo4j AuraDB + GraphXR, we have successfully built a new information system design that can inform the shift in demand for skills, and skill clusters over time. To replicate feasible analytical solutions offered by existing job portals in the market we used AWS data warehouse solutions.
Future Work:
We believe that the integration of the employee review data into our information system design will allow us to better understand how employee perceptions have changed over time by industry, sector, employer, region, etc. Understanding employee views is crucial for tracking the state of the economy and business over time.
