Sr. Splunk Engineer

Come build the next-gen fintech at Early Warning, network operator of Zelle®, where we're relentlessly focused on empowering prosperity in all its forms.

From fast money movement for over 100 million people who can access Zelle® directly through their banking app to new account opening and beyond - we make a difference in the lives of consumers and businesses every day and enable them to live their best financial lives. And we're only getting started.

With new state-of the-art offices in Scottsdale, AZ (Headquarters) and Chicago, IL - plus a growing presence in San Francisco - we're entering our next big chapter.

People matter to us, so we think our best work is done when we're together, in-person. From informal interactions, to sharing ideas, to mentoring and beyond. We also believe in workplace flexibility and empowering teams to determine the rhythm of work to create an inclusive culture. Through the power of innovative collaboration, we offer hybrid and (when necessary) virtual ("remote") workplace models.

We focus all hiring efforts in a few states and within driving distance of an office location to enable in-person collaboration when and where possible. Priority hiring locations are Scottsdale, AZ, Illinois, NY tri-state metro area (New Jersey, New York and Connecticut) and San Francisco, CA. On exception we will hire in secondary locations such as District of Columbia, Florida, Georgia, Maryland, Nevada, North Carolina, South Carolina, Texas and Virginia. We are not actively recruiting in Colorado, Rhode Island or Washington.

Join us and make your mark on what's next in fintech.

*Applicants must be authorized to work for any employer in the United States. We are unable to sponsor an employment Visa for this position.

Overall Purpose

The Senior Splunk Engineer will support Production Assurance and Reliability Operations Services and ensure that the company operates a world-class enterprise log management solution.

The Senior Splunk Engineer maintains the logging infrastructure for Splunk Enterprise and other dependent infrastructure, such as syslog (syslog-ng/rsyslog) to ensure that reports are available, fast and efficient. They partner with stakeholders to gather and analyze user needs; evaluate and onboard new data sources; and ensure the operational and security integrity of the infrastructure. In addition, they will assist end users with searches, dashboards, reports, and knowledge objects. The Senior Splunk Engineer leads and/or consults on efforts pertaining to design, architecture, scalability, business continuity, and disaster recovery as needed.

Essential Functions

• Oversees Splunk Enterprise infrastructure and tunes Splunk for optimal onboarding of data, performance, and capacity management; identifies gaps and areas of duplication; provides recommendations for optimization
• Manages and maintains a complex multi-site hybrid environment and ensures the infrastructure remains available and scalable. Provides technical expertise in the maintenance and expansion of IT architecture to support business needs such as software additions, large-scale data stores and cloud -based systems
• Ensures Splunk is monitored, and effective alerting is in place to meet SLAs and required visibility
• Manages upgrades and patching of platform to ensure EWS identified vulnerabilities are mitigated or brought to resolution, and the infrastructure remains within supported guidelines
• Ensures our logging environment provides for effective threat detection and response in direct partnership with information security teams
• Participates in incident management & incident response during an outage or breach when needed
• Interact with internal/ external customers to gather requirements, perform troubleshooting and aid with the creation of Splunk search queries. Designs and develops reports and dashboards in Splunk. Diagnoses, solves, and implements needed solutions for a complex environment.
• Creates documentation for any addition or change to our environment. Reviews and updates on a regular basis to ensure accuracy.
• Ensures the security tools are supporting all compliance efforts in collaboration with auditors
• Provides metrics for platform performance, capacity, and user management
• Assists with compliance questionnaires as needed
• Collaborates with the Security team in security incident resolution and risk evaluations
• On call rotation with other Splunk engineers to cover 24x7 response
• Sets and supports best practices for end users and company standards. Stays current on the latest industry technologies, trends, and strategies
• Support the company's commitment to protect the integrity and confidentiality of systems and data.

Minimum Qualifications

• This position requires a high school diploma or GED. Additional college level Computer Science, Engineering or business-related classes are beneficial.
• Minimum 5 years of progressive Splunk administration, Splunk architect and/or logging experience in a multisite environment is necessary
• Certification: Splunk Certified Administrator required
• Practical experience with major components of Splunk Enterprise, including data ingestion pipelines, and SPL
• Solid Linux and Windows foundation with an emphasis of Unix/Linux CLI
• Demonstrated proficiency with the full Splunk lifecycle, including all major components for an enterprise deployment.
• Must have solid foundation in Linux and possess a competence to troubleshoot various aspects of the integration including operating system, application, and networking components as they relate to both Splunk and syslog implementations
• Ability to handle large projects as well as take care of day-to-day operations
• Strong sense of responsibility, ownership, and pride in delivering quality results.
• In addition, troubleshooting and organizational skills with a can-do attitude and the ability to adjust to changing requirements are essential.
• Demonstrated ability to communicate across all levels of the organization is necessary; must be able to clearly articulate technical ideas to a non-technical audience both verbally and in writing.
• Background and drug screen

Preferred Qualifications

• Bachelor's degree in Computer Science, Engineering, Math or Physical Science.
• Additional related education and/or experience preferred
• Common ingestions strategies into Splunk such as UFs, HFs, HEC, TAs/Add-ons, syslog, etc.
• Shell/Interpreter scripting a plus (e.g., Bash, PERL, Python)
• Certifications: Splunk Core Certified Advanced Power User, Splunk Enterprise Certified Architect (working towards or achieved strongly desired)
• Unix/Linux administration background a strong plus
• Experience with Splunk ES / UBA
• Logging technologies (implementation/administration) - syslog-ng/rsyslog, ELK, Filebeat, Kafka, Cribl, Fluentd, Graylog
• X.509/PKI fundamentals with experience on practical implementation
• Practical understanding and troubleshooting experience with the following: UDP, TCP, SSL/TLS, HTTP/S, Rest APIs, load balancers, forward proxies

Physical Requirements

Working conditions consist of a normal office environment. Work is primarily sedentary and requires extensive use of a computer and involves sitting for periods of approximately four hours. Work may require occasional standing, walking, kneeling, and reaching. Must be able to lift 10 pounds occasionally and/or negligible amount of force frequently. Requires visual acuity and dexterity to view, prepare, and manipulate documents and office equipment including personal computers. Requires the ability to communicate with internal and/or external customers.

Employee must be able to perform essential functions and physical requirements of position with or without reasonable accommodation.

The above job description is not intended to be an all-inclusive list of duties and standards of the position. Incumbents will follow instructions and perform other related duties as assigned by their supervisor.

Early Warning Services is an affirmative action and equal opportunity employer.

The pay scale for this position in:

Phoenix, AZ/ Chicago, IL in USD per year is: $120,000 - $140,000.

New York, NY in USD per year is: $130,000 - $160,000.

This pay scale is subject to change and is not necessarily reflective of actual compensation that may be earned, nor a promise of any specific pay for any specific candidate, which is always dependent on legitimate factors considered at the time of job offer. Early Warning Services takes into consideration a variety of factors when determining a competitive salary offer, including, but not limited to, the job scope, market rates and geographic location of a position, candidate's education, experience, training, and specialized skills or certification(s) in relation to the job requirements and compared with internal equity (peers). The business actively supports and reviews wage equity to ensure that pay decisions are not based on gender, race, national origin, or any other protected classes.

Additionally, candidates are eligible for a discretionary bonus, and benefits.

Some of the Ways We Prioritize Your Health and Happiness

• Healthcare Coverage - Competitive medical (PPO/HDHP), dental, and vision plans as well as company contributions to your Health Savings Account (HSA) or pre-tax savings through flexible spending accounts (FSA) for commuting, health & dependent care expenses.

• 401(k) Retirement Plan - Featuring a 100% Company Safe Harbor Match on your first 6% deferral immediately upon eligibility.

• Paid Time Off - Unlimited Time Off for Exempt (salaried) employees, as well as generous PTO for Non-Exempt (hourly) employees, plus 11 paid company holidays and a paid volunteer day.

• 12 weeks of Paid Parental Leave

• Maven Family Planning - provides support through your Parenting journey including egg freezing, fertility, adoption, surrogacy, pregnancy, postpartum, early pediatrics, and returning to work.

And SO much more! We continue to enhance our program, so be sure to check our Benefits page here for the latest. Our team can share more during the interview process!

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Early Warning Services, LLC ("Early Warning") considers for employment, hires, retains and promotes qualified candidates on the basis of ability, potential, and valid qualifications without regard to race, religious creed, religion, color, sex, sexual orientation, genetic information, gender, gender identity, gender expression, age, national origin, ancestry, citizenship, protected veteran or disability status or any factor prohibited by law, and as such affirms in policy and practice to support and promote equal employment opportunity and affirmative action, in accordance with all applicable federal, state, and municipal laws. The company also prohibits discrimination on other bases such as medical condition, marital status or any other factor that is irrelevant to the performance of our employees.