Job Overview Job Description:Join AT&T and reimagine the communications and technologies that connect the world. Our Software Resiliency and Reliability Engineering Technology organization is delivering innovative and reliable technology solutions for AT&T mission critical applications to trace business functions across multiple distributed systems. Bring your bold ideas and fearless risk-taking to redefine connectivity and transform how we connect the world and deliver unsurpassed customer experiences. When you step into a career with AT&T, you won't just imagine the future-you'll create it.Here at AT&T, the SRE organization focuses on providing uninterrupted customer experience through improving the resiliency and reliability of all components in a customer journey. The program identifies critical customer journeys across all organizations within AT&T and takes a two-prong approach by conducting both Static and Dynamic analysis. The Static Analysis uses manual assessments to identify initial resiliency score and maturity level. Dynamic Analysis requires instrumenting automated resiliency frameworks including Rainmaker, T-trace, T-view and, MechID/SSL Bot to collect deep insights of the application behavior. It is a very exciting, visible, and growing area that requires technical expertise and thought leadership to drive these efforts across AT&T enterprise ecosystem.The position on the SRE T-trace team leverages an open-source industry standard Open Telemetry product that is the standard tracing solution for AT&T. T-Trace provides full visibility of end-to-end business transactions across multiple distributed systems Applications send structured tracing data to global collection points enabling view of actual live service flows. It allows operations teams to quickly identify root cause of outage and engage correct resolution groups to reduce operational mean time to identify (MTTI). With more and more applications onboarded with T-trace, its value will accelerate exponentially.This career step requires Lead level experience. Responsible for the overall delivery process to instrument T-trace OTEL agent on AT&T mission critical applications, including design, development, testing, deployment, production application support (e.g., troubleshooting using T-trace if the application is involved in an outage). Key Roles and Responsibilities: Collaborate with the application technical lead to define the T-trace architecture solution and appropriate patterns for instrumentation. Work with the application teams to install the OTEL agent and make configurations changes according to the SRE architecture solution design in all non-prod environments. Develop custom instrumentation solutions based on the application's programming language or architecture. Collaborate with Expert engineers/architects to resolve technical issues. Integrate the Open Telemetry agent and configuration within their respective CI/CD flows. Collaborate with development teams, clients, etc. to validate test data and test results, design and implement code fixes, validate outcomes to ensure meaningful trace data. Evaluate performance testing in Test/QA environments, calculate expected trace span load for production environment, validate tracing data and performance impact for production readiness. Perform postproduction validation to ensure successful implementation of Open Telemetry solution in the production environment and demonstrate benefits from trace data in the Service Now Cloud Observability Tool." Job Contribution: Lead level technical expertise. Deep technical knowledge and subject matter expert on AT&T technologies.Required Skills: -Strong Java and Related TechnologyExposure to most of the popular java frameworks and libraries in heavy use since 2010Understanding of JVM internals and profiling, JVM memory and garbage collection tuning and heap dump analysisExperience with java multi-threading and the evolution of the language in this area.Ability to operate as a code contributor to the open telemetry java instrumentation projectRESTful web service and SOAP web servicesExperience with cloud computing including successfully developing and deploying applications to the cloud such as AWS, GCP, AzureExposure to Go including the ability to read code written in Go and write Go code making use of the Go paradigms such as go routine and familiar with many Go system librariesExperience with CI/CD and infrastructure as codeCreative Thinking and Problem SolvingAble to work and collaborate effectively with multiple application teams and vendor partnersThought leadership (solutioning through creative/new paths)Resourcefulness (research alternative approaches)Prior distributed tracing expertise with java applications would be helpfulEducation: Preferred Bachelor of Science degree in Computer Engineering, Computer Science, Applied Science, Electrical Engineering, or Math; Developer nanodegree; or equivalent experience. Experience: 5-8 years' experienceOur Lead Software Engineering earn between $143,800 -$215,800. Not to mention all the other amazing rewards that working at AT&T offers. Individual starting salary within this range may depend on geography, experience, expertise, and education/training.Joining our team comes with amazing perks and benefits: Medical/Dental/Vision coverage 401(k) plan Tuition reimbursement program Paid Time Off and Holidays (based on date of hire, at least 23 days of vacation each year and 9 company-designated holidays) Paid Parental Leave Paid Caregiver Leave Additional sick leave beyond what state and local law require may be available but is unprotected Adoption Reimbursement Disability Benefits (short term and long term) Life and Accidental Death Insurance Supplemental benefit programs: critical illness/accident hospital indemnity/group legal Employee Assistance Programs (EAP) Extensive employee wellness programs Employee discounts up to 50% off on eligible AT&T mobility plans and accessories, AT&T internet (and fiber where available) and AT&T phone.AT&T is leading the way to the future - for customers, businesses and the industry. We're developing new technologies to make it easier for our customers to stay connected to their world. Together, we've built a premier integrated communications and entertainment company and an amazing place to work and grow. Team up with industry innovators every time you walk into work, creating the world you always imagined. Ready to #transformdigital with us? Apply nowWeekly Hours:40Time Type:RegularLocation:USA:TX:Dallas / 208 S Akard St - Concourse:208 S Akard St @ ConcourseSalary Range: $128,400.00 - $215,800.00It is the policy of AT&T to provide equal employment opportunity (EEO) to all persons regardless of age, color, national origin, citizenship status, physical or mental disability, race, religion, creed, gender, sex, sexual orientation, gender identity and/or expression, genetic information, marital status, status with regard to public assistance, veteran status, or any other characteristic protected by federal, state or local law. In addition, AT&T will provide reasonable accommodations for qualified individuals with disabilities. Job ID R-2885-2 Date posted 02/21/2024