What are the responsibilities and job description for the Senior DevOps Engineer position at PolySwarm?
Sr DevOps Engineer at PolySwarm - 100% remote, in continental US
Join the Swarm
We are looking for a talented Senior DevOps Engineer to join our team!
PolySwarm is developing innovative solutions to age-old information security problems - and we need your help. The PolySwarm marketplace is a launchpad for new technologies and novel threat detection methods, where commercial solutions and independent researchers compete to detect threats, and get compensated based on performance.
At PolySwarm, you'll join a tight, dynamic team on the bleeding edge of information security, answering questions few have thought to ask. As an information security company, we use current technologies and do our best to regularly upgrade our infrastructure to take advantage of improvements in security, reliability, and scaling.
On a typical day you will be helping us design, architect, and refine our software and infrastructure to support new features and services. Making improvements to our services and infrastructure to handle vast amounts of data through the full pipeline of transmission, processing, and storage. Evaluating technologies and mechanisms to optimize storage and query performance. Finding optimal ways to authenticate and process millions of queries per day. Building and improving our CI/CD processes and architectures.
When it is time to build new services or major features, you may be the main architect, and even if you are not, we will definitely ask for your input into the design and implementation details. We are a small team and share a diverse set of tasks across the team, so you’ll be able to have an impact anywhere you show interest.
With your expertise and experience, you will play a critical role in making PolySwarm the most competitive anti-malware platform available. Senior hires will be entrusted with broad decision-making responsibility concerning architecture, infrastructure, deployment, scale, access control, continuous integration, tooling, and much, much more.
If you are an ambitious, seasoned DevOps engineer interested in creating solutions to the most challenging and pressing security challenges of the moment, we are interested in speaking with you.
Current Real-World Challenges
We need to process and store millions of files per day and that number is growing. When we say process, we mean that a large collection of tools need to analyze each file to produce metadata about that file. How do we store that much metadata to make it efficiently searchable? How do we rapidly scale up and down as the stream of files and searches changes throughout the day/week?
Explore the PolySwarm marketplace here: https://polyswarm.network/
In compliance with U.S. federal law, all persons hired who live in the U.S. will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification form upon hire.
PolySwarm provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
Requirements:
What you’ll need to be successful:
Join the Swarm
We are looking for a talented Senior DevOps Engineer to join our team!
PolySwarm is developing innovative solutions to age-old information security problems - and we need your help. The PolySwarm marketplace is a launchpad for new technologies and novel threat detection methods, where commercial solutions and independent researchers compete to detect threats, and get compensated based on performance.
At PolySwarm, you'll join a tight, dynamic team on the bleeding edge of information security, answering questions few have thought to ask. As an information security company, we use current technologies and do our best to regularly upgrade our infrastructure to take advantage of improvements in security, reliability, and scaling.
On a typical day you will be helping us design, architect, and refine our software and infrastructure to support new features and services. Making improvements to our services and infrastructure to handle vast amounts of data through the full pipeline of transmission, processing, and storage. Evaluating technologies and mechanisms to optimize storage and query performance. Finding optimal ways to authenticate and process millions of queries per day. Building and improving our CI/CD processes and architectures.
When it is time to build new services or major features, you may be the main architect, and even if you are not, we will definitely ask for your input into the design and implementation details. We are a small team and share a diverse set of tasks across the team, so you’ll be able to have an impact anywhere you show interest.
With your expertise and experience, you will play a critical role in making PolySwarm the most competitive anti-malware platform available. Senior hires will be entrusted with broad decision-making responsibility concerning architecture, infrastructure, deployment, scale, access control, continuous integration, tooling, and much, much more.
If you are an ambitious, seasoned DevOps engineer interested in creating solutions to the most challenging and pressing security challenges of the moment, we are interested in speaking with you.
Current Real-World Challenges
We need to process and store millions of files per day and that number is growing. When we say process, we mean that a large collection of tools need to analyze each file to produce metadata about that file. How do we store that much metadata to make it efficiently searchable? How do we rapidly scale up and down as the stream of files and searches changes throughout the day/week?
- Our users send millions of API requests per day to our platform. How do we continue to scale to support the authentication of those requests? How do we continue to scale to support answering those requests?
- Our databases have billions of rows in some tables. How do we structure the databases to support efficient migrations, backups, and queries? Do you know the table size limitations of data storage technologies?
- We have a feature called Historical Hunting where we run on-demand jobs that execute customer-provided YARA rules over millions of files. How do we continue to scale that in a cost effective way?
- Competitive salaries
- Excellent health, dental, vision coverage (US employees)
- 401k (US employees)
- Paid time off (US employees)
- Flexible work hours. All staff must live and work in the continental US. We have core work hours for overlap between time zones, but outside of that block of hours, you can schedule your working hours as you like to get things done.
- Powerful laptops and desktops - whatever you need to be most productive!
Explore the PolySwarm marketplace here: https://polyswarm.network/
In compliance with U.S. federal law, all persons hired who live in the U.S. will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification form upon hire.
PolySwarm provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
Requirements:
What you’ll need to be successful:
- MS or PhD in Computer Science, Software Engineering, or related field
- 5 years production software engineering skills building microservices with Python in a cloud architecture
- 10 years production software architecture/design experience
- 5 years experience with production cloud infrastructure in AWS managed by Kubernetes
- Experience managing high-speed transactions in relational databases
- Experience managing large volumes of data in Elasticsearch or similar technologies
- Experience processing and storing large datasets
- Experience building ETL processes
- Experience with infrastructure monitoring and alerting technologies
- Experience with docker
- Experience with firewalls, ACLs, SSO, and Oauth
- Experience designing and managing CI/CD for a large set of microservices
- Comfortable working in a remote-only environment
- Analytical mind
- Confidence to share your ideas
- Ability to clearly describe and document your ideas
- Ability to constructively critique your ideas and those of your team members
- Ability to independently motivate & self-direct
- Ability to collaborate and work in a team
- Proficient in English (reading, writing, and speaking)
- Background in Computer security and Network security related fields
- Experience with big data analytics
- Experience with the infrastructure side of financial markets, high-frequency trading, or cryptocurrency
- AWS
- RabbitMQ
- Kafka
- Kubernetes
- Helm
- NGINX
- Elastic Search
- Linux
- Redis
- Postgresql
- GitLab
- GitHub
- Kibana
- Python
- Docker/Docker Compose
- Blockchain/Ethereum
- Rust
- WINE
