Cloud Data Engineer (SQL, AI Search, and Observability)

Date: 10.2025

Function: Engineering

Reports to: Director Platform Engineering

Location: Remote United States

Position Summary

As a Cloud Data Engineer you will be at the intersection of traditional database engineering modern AI search and advanced observability practices. This role is ideal for someone who thrives in hybrid environments—combining deep SQL expertise with modern search platforms like Azure AI Search and ElasticSearch. OnBoard is looking for someone who is excited about building intelligent observable data systems for LLM-powered applications.

You’ll work across structured data platforms real-time indexing pipelines and AI-enhanced retrieval systems ensuring reliability performance discoverability and security. You’ll also contribute to the infrastructure that supports observability for search relevance query performance and LLM pipeline diagnostics.

Key Responsibilities

• Design deploy and operate Microsoft SQL Server workloads across Azure IaaS (VM) and PaaS (Azure SQL DB / SQL Managed Instance).

• Build scale and tune AI Search and ElasticSearch clusters pipelines and relevance models for both keyword and semantic search.

• Collaborate with developers and ML teams to build retrieval pipelines for LLM applications including vector indexing prompt metadata enrichment and embeddings.

• Integrate observability into all layers of the data/search stack.

• Analyze and optimize SQL queries and indexing strategies using Query Store DMVs and execution plans.

• Secure data access using encryption (TDE Always Encrypted) private endpoints and RBAC.

• Automate recurring operations using PowerShell/Azure DevOps Pipelines including job scheduling alerting index maintenance and schema validation.

• Implement Infrastructure-as-Code (IaC) for SQL and search platforms using Bicep.

• Drive incident resolution and root cause analysis for issues across SQL search and LLM pipelines.

• Maintain technical documentation and operational runbooks for the combined SQL + Search + AI infrastructure.

Skills and Experience Needed

• 3–5 years of experience managing Microsoft SQL Server in production both IaaS and PaaS.

• Strong experience with full-text and semantic search using Azure Cognitive Search and/or ElasticSearch.

• Experience building or maintaining RAG/LLM architectures including vector stores embedding generation and retrieval APIs.

• Proficiency in tuning and debugging database performance (wait stats execution plans indexing).

• Hands-on experience with observability platforms and standards (OpenTelemetry Grafana APM tools etc.).

• Automation scripting with PowerShell or Python.

• Proficiency with Infrastructure-as-Code (Bicep Terraform ARM templates).

• Familiarity with compliance RBAC and secure data practices in cloud environments.

• Bonus: experience instrumenting and debugging LLM inference chains or vector database retrieval quality.

Competencies

• Accountability

• Adaptability

• AI Aptitude

• Applied Learning

• Business Acumen

• Collaboration

• Customer Focus

• Dealing w/Ambiguity

• Decision Making

• Driving for Results

• Initiating Action

• Planning & Organizing

• Technical/Professional Knowledge/Skills