Gen AI Content Evaluator and Team Lead

Introduction
At IBM work is more than a job – it’s a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better but to attempt things you’ve never thought possible. Are you ready to lead in this new era of technology and solve some of the world’s most challenging problems? If so lets talk

Your Role and Responsibilities
Are you interested in working in one of the most impactful areas of technology in the world today? Do you want to build generative AI skills while working on a project to transform the most mission-critical IT workloads for organizations that power the global economy? Come join the team that is at the intersection of cutting-edge gen AI and mainframe software development a key strategic pillar for IBM.

Roles and responsibilities include:

• Team lead responsible for the evaluating and benchmarking the LLM and fine tune its performance.
• This may include contributions to the automated evaluation pipeline in addition to advancing techniques used to evaluate performance as well as metrics.
• In your role as architect you will contribute to the design of the evaluation and benchmarking strategy and provide ongoing analysis and metrics on current performance.
• You will be responsible to incrementally improving our measurement and analysis over time.

Required Technical and Professional Expertise

• Outstanding communication and collaboration skills
• Strong technical architecture experience in addition to experience leading a team
• Minimum of 10 years of recent industry experience programming in COBOL PL/I Assembler JCL and/or REXX
• Demonstrated ability to work with remote teams

Preferred Technical and Professional Expertise

• Any prior Java programming experience
• Experience with Agile software development
• Automation skills such as Ansible/Python).
• knowledge of z/OS