Data Engineer - AWS/ETL/Python

Introduction
At IBM work is more than a job – it’s a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better but to attempt things you’ve never thought possible. Are you ready to lead in this new era of technology and solve some of the world’s most challenging problems? If so lets talk.

Your Role and Responsibilities
Day-to-day troubleshooting of forecasting systems mainly working through data anomalies that cause inaccurate forecasts or prevent forecasts’ generation.
Collaborate with the data science team to enhance existing forecasting systems for the trade floors.
Create dynamic object-oriented methods full stack solutions and integrations to existing code solutions.
Develop individual Python classes methods functions that support the data flow of existing and new projects.
Work on code additions to seamlessly support projects for data flows including logging and support with little to no supervision.
Experience in modifying packages testing and repository instances to support CI/CD.

Required Technical and Professional Expertise
Design develop test and deploy Python applications on AWS ensuring high availability scalability and security
Develop and maintain technical documentation for Python applications and AWS infrastructure
Optimize application performance scalability and reliability using AWS services such as:
+ Serverless technologies (Lambda API Gateway Step Functions)
+ S3 for data storage and retrieval
+ Glue for data integration and ETL
+ SQS and SNS for message queuing and notification
+ Cognito for user authentication and authorization
+ CloudWatch for monitoring and logging

Preferred Technical and Professional Expertise
Implement automated testing deployment and monitoring using tools like Jenkins Docker and CloudWatch
Troubleshoot and resolve technical issues in Python applications and AWS infrastructure
Stay up-to-date with the latest developments in Python and AWS and apply this knowledge to improve our applications and infrastructure
Design and develop Serverless front-end applications using AWS services such as API Gateway Lambda and S3
Implement PDS Proxy to handle data processing and analytics workloads
Integrate AWS services with external systems and APIs using APIs SDKs and other integration tools
Implement Row Level Security (RLS) to segregate data and ensure secure access to sensitive information
Design and implement data models and database schemas for DynamoDB and PostgreSQL
Design and develop data ingestion pipelines using AWS Lambda Glue AWS Batch and restful API’s
Thorough understanding of AWS cloud concepts and related technologies like AWS VPC Subnets AZ’s SG IAM policies & Roles EC2 AWS ALB API Gateway ECS RDS AWS MSK Kinesis SQS SNS S3 DynamoDB Secret Manager Cloud Watch Cloud Formation scrips(yaml/json) AWS CDK.
Collaborate with cross-functional teams to identify and prioritize project requirements
Knowledge of API unit & performance testing tools like postman JMeter SOAP UI is a plus.
Stay up to date with emerging technologies and trends evaluating their potential impact on our projects.
Mentor and provide technical guidance to junior developers fostering their growth and helping them overcome challenges.
Collaborate with cross-functional teams to integrate web applications with existing systems and third-party services.
Ensure compliance with development standards security guidelines and best practices.
Conduct code reviews to maintain high code quality consistency and adherence to coding standards.