New York Life Insurance Company

Senior Associate - Elasticsearch Engineer

Posted 3 Hours Ago

Be an Early Applicant

New York NY USA

In-Office

124K-177K Annually

Senior level

Artificial Intelligence • Cloud • Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics

The Role

The Elasticsearch Engineer manages Elasticsearch clusters develops ingest pipelines ensures platform performance and collaborates with security data teams to meet compliance and retention requirements.

Summary Generated by Built In

Location Designation: Hybrid - 3 days per quarter
Role Overview
As part of Technology you'll have the opportunity to contribute to groundbreaking initiatives that shape New York Life's digital landscape. Leverage cutting-edge technologies like Generative AI to increase productivity streamline processes and create seamless experiences for clients agents and employees. Your expertise fuels innovation agility and growth - driving the company's success.
The Elasticsearch Engineer will work on a team supporting the day-to-day operation performance and reliability of the enterprise Elasticsearch platform that serves as the core of our security data lake. This is a mid-level engineering role (3-5 years of experience) embedded within the Security Data Platform team focused on Elasticsearch cluster operations ingest pipeline development and data architecture.
What You'll Do:
The engineer will be responsible for managing a large Elasticsearch 9.x cluster environment building and maintaining ingest pipelines that normalize high-volume security log sources to Elastic Common Schema (ECS) designing and tuning index templates and data stream lifecycles across hot/warm/cold/frozen tiers and ensuring the platform meets performance SLAs and compliance-driven retention requirements. This role works closely with the log routing layer (Cribl) and the SOC engineering team.
What You'll Bring

Hands-on experience operating and troubleshooting multi-node Elasticsearch clusters (40+ nodes) including shard allocation recovery tuning backpressure diagnosis and node-level resource management
Strong understanding of Index Lifecycle Management (ILM) policies across hot/warm/cold/frozen tiers including searchable snapshots and frozen-tier index restoration workflows
Experience building and maintaining ingest pipelines using native Elasticsearch processors (grok set rename convert script pipeline chaining) with a preference for processor-based approaches over Painless where possible
Working knowledge of Painless scripting for ingest-time field transformations conditional logic and data normalization
Proficiency with index templates component templates and data stream architecture - including understanding of mapping conflicts dynamic templates and failure store indices
Familiarity with Elastic Common Schema (ECS) field mapping conventions and how to apply them to security log sources during ingest
Experience with data stream rollovers reindexing operations and mapping migration strategies for live production data
Ability to write and optimize ES|QL and KQL queries for security use cases and build/maintain Kibana dashboards and data views
Experience monitoring and tuning search performance including slow query log analysis shard sizing strategies query profiling and understanding the impact of mapping choices (keyword vs text doc_values subobjects) on query efficiency
Familiarity with cluster health and performance monitoring via Kibana Stack Monitoring and Devtools for diagnosing allocation and performance issues

Preferred / Nice-to-Have

Experience with cross-cluster search (CCS) and remote cluster configuration in multi-cluster architectures
Familiarity with Terraform-managed Elasticsearch resources (roles API keys index templates data views)
Exposure to Cribl Stream or similar log routing/transformation platforms feeding into Elasticsearch via HEC or Elasticsearch output
Understanding of compliance-driven data retention requirements (e.g. NY DFS NAIC) and how they map to ILM/tier policies
Experience with Elastic Security app detection rules or security-focused Kibana content
Experience with Elastic Cloud cost management including deployment sizing autoscaling behavior data tier cost optimization (hot vs frozen storage economics) and identifying savings opportunities through shard consolidation ILM tuning or field reduction at ingest
Understanding of capacity planning - forecasting storage and compute needs based on ingest rates retention requirements and query workload patterns

#LI-CD2
Pay Transparency
Salary Range: $124000-$177000
Overtime eligible: Exempt
Discretionary bonus eligible: Yes
Sales bonus eligible: No
Actual base salary will be determined based on several factors but not limited to individual's experience skills qualifications and job location. Additionally employees are eligible for an annual discretionary bonus. In addition to base salary employees may also be eligible to participate in an incentive program.
Company Overview
At New York Life our 180-year legacy of purpose and integrity fuels our future. As we evolve into a more technology- data- and AI-enabled organization we remain grounded in the values that drive lasting impact.
Our diverse business portfolio creates opportunities to make a difference across industries and communities-inviting bold thinking collaborative problem-solving and purpose-driven innovation. Here you'll find the rare balance of long-standing stability and forward momentum supported by an inclusive team that honors tradition while embracing progress.
As a Fortune 100 mutual company we offer a place to grow your skills contribute to meaningful work and deliver solutions that matter. Your ideas drive what's next and your growth powers it.
Our Benefits
We provide a full package of benefits for employees - and have unique offerings for a modern workforce including leave programs adoption assistance and student loan repayment programs. Based on feedback from our employees we continue to refine and add benefits to our offering so that you can flourish both inside and outside of work.Click hereto discover more about our comprehensive benefit options or visit our NYL Benefits Site.
Our Commitment to Inclusion
At New York Life fostering an inclusive workplace is fundamental to who we are and how we serve our communities. We have a longstanding commitment to creating an environment where individuals can contribute their best and succeed together. This foundation is rooted in our core values of humanity and integrity ensuring that every employee feels valued and supported. By embracing a broad range of perspectives and experiences we achieve greater success and fulfill our promise of providing financial security and peace of mind to families across all communities. Click here to learn more about New York Life's leadership in this space.
Recognized as one of Fortune's World's Most Admired Companies New York Life is committed to improving local communities through a culture of employee giving and volunteerism supported by the Foundation. We're proud that due to our mutuality we operate in the best interests of our policy owners. To learn more about career opportunities at New York Life please visit the Careers page of www.NewYorkLife.com.
Visit our LinkedIn to see how our employees and agents are leading the industry and impacting communities.
Visit our Newsroom to learn more about how our company is constantly evolving to meet our clients' and employees' needs.
Job Requisition ID: 93721

Top Skills

Cribl

Elasticsearch

Generative Ai

Kibana

Terraform

What the Team is Saying

View all jobs at New York Life Insurance Company

View New York Life Insurance Company Profile

Report Job

Am I A Good Fit?

beta

Get Personalized Job Insights.

Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company

HQ: New York NY

12000 Employees

Year Founded: 1845

What We Do

At New York Life our 180-year legacy of integrity mutuality and financial strength fuels a future defined by bold transformation. As the largest mutual life insurance company in the U.S. we operate on behalf of our policy owners—not shareholders. That structure allows us to take a long-term view investing in people purpose and innovation that endures. Guided by a clear enterprise vision to become a technology- data- and AI-powered company we’re modernizing our platforms rearchitecting experiences and embedding intelligence across our products and services. Our mission has always been about helping people through life’s most meaningful moments. Today technology is amplifying that mission—enabling us to serve clients advisors and communities in more personalized proactive ways. With a diversified business portfolio spanning insurance investments retirement group benefits and direct-to-consumer offerings New York Life delivers the stability of a Fortune 100 company with the agility of one that’s continuously evolving. We’re powered by a values-led culture inclusive teams and a shared belief that when our people thrive so does our company. Here tradition fuels momentum—and your ideas energy and growth power what’s next.

Why Work With Us

New York Life is transforming from the inside out—blending 180 years of trust with the velocity of innovation. What makes us different is our culture: grounded in integrity humanity and shared success—values that show up in how we work lead and grow. If you want a place where innovation has purpose—build what's next with us.