Python Developer/ Application Support Engineer - W2 only

Job Title: Python Developer / Application Support Engineer

Location: Location: 100% Onsite – Cambridge, MA, Lawrenceville, NJ, San Diego, CA, Seattle, WA

Job ID # 81649

Rate type: W2 only

Top Skills:

• Proficiency in programming languages such as Python, SQL, R, and JavaScript.

• Experience developing web applications in frameworks like Streamlit, Shiny, Vue, React, etc.

• Proven experience with core AWS services, including but not limited to EC2, S3, and RDS.

• BioPharma experience preferred

Note: The interview process for this role will require a real-time coding assignment.

Position Summary:

We are seeking a dedicated and skilled LLM Application Support Engineer to join our technical team responsible for providing L1, L2, and L3 specialized application support for Large Language Model (LLM)-based applications. The successful candidate will also be responsible for implementing automated testing frameworks for both applications and LLMs specifically. This role requires a solid understanding of LLMs, application support, and automated testing methodologies.

Responsibilities include, but are not limited to, the following:

Application Support:

• Provide L1, L2, and L3 support for LLM-based applications, ensuring timely resolution of issues.

• Ensure the scalability and performance of LLM-based applications to handle large datasets and complex queries.

• Identify opportunities for process improvements and implement innovative solutions to enhance the efficiency and effectiveness of LLM deployments.

• Monitor application performance and proactively identify potential issues.

• Troubleshoot and resolve application issues, escalating to higher levels of support when necessary.

• Collaborate with development teams to address and resolve complex technical issues.

• Maintain detailed documentation of support activities, including issue resolution steps and best practices.

• Ensure compliance with BMS's data security and regulatory requirements during support activities.

• Conduct root cause analysis for recurring issues and implement preventive measures.

Automated Testing Frameworks:

• Develop and implement automated testing frameworks for LLM-based applications.

• Design and execute automated tests to validate the functionality, performance, and reliability of applications and LLMs.

• Create and maintain test scripts for unit tests, integration tests, and end-to-end tests.

• Collaborate with development teams to integrate automated testing into the CI/CD pipeline.

• Monitor and analyze test results, identifying and reporting defects.

• Continuously improve automated testing processes and frameworks to enhance test coverage and efficiency.

• Ensure compliance with BMS's data security and regulatory requirements during testing activities.

• Stay updated with the latest trends and best practices in automated testing and LLM technologies.

Basic Qualifications:

• Bachelor's Degree with 2-4 years of academic/industry experience in application support or development.

• Preference for candidates with experience in machine learning or LLM-based applications.

Preferred Qualifications:

Technical:

• Proven experience with core AWS services, including but not limited to EC2, S3, and RDS.

• Demonstrated proficiency with current software engineering methodologies, such as Agile SDLC approaches, distributed source code control, project management, issue tracking, and CI/CD tools and processes.

• Proficiency in programming languages such as Python, SQL, R, and JavaScript.

• Strong understanding of machine learning frameworks and libraries (e.g., TensorFlow, PyTorch, scikit-learn).

• Solid understanding of container strategies such as Docker and ECS.

• Excellent skills and deep knowledge of databases such as Postgres, Elasticsearch, and Redshift, including distributed database design, SQL vs. NoSQL, and database optimizations.

• Experience developing web applications in frameworks like Streamlit, Shiny, Vue, React, etc.

• Proficiency with predictive modeling approaches and/or experience working with large language models.

• Experience with testing frameworks and automated testing tools (e.g., pytest, Selenium, JUnit).