Automating My Job Search with AWS Lambda, DynamoDB and Google Sheets
27 Aug 2023Elaborating on the goal
As a computer science student preparing to graduate soon, I wanted to build a system to automatically find and track relevant job openings tailored to my skills and interests. The main goal was to create an automated pipeline that would continuously scrape job sites for opportunities and collate them in a centralized location so I can easily track applications and deadlines.
Explaining the tech stack
I decided to use AWS Lambda functions for the scraper logic since it allows running code on a schedule without managing servers. DynamoDB provides a fast and scalable datastore for capturing job listings as they are found. I’m using SQS queues to efficiently pass data between the Lambda functions. Finally, the collated job listings are populated into a Google Sheet using the Sheets API for a clean and shareable output.
To Summarize:
- AWS Lambda
- DynamoDB
- AWS SQS(simple Queing Service)
- Google Sheets API
I have a combination of two lambda functions running:
-
To Fetch new jobs every few hours and push them to DynamoDB
-
To Fetch newly inserted rows in DynamoDB and push them to Google Sheets
Link to the Jobs being found: (Staying on Top of the Job Hunt with a Serverless Job Listings Tracker)
Next Steps:
- To Add Experience to make it more relevant to people applying
- Add Top skills required by the Job, eg: Deep Learning, Scikit-learn