Job Title: (Software Engineer/Developer)
We are looking for an experienced Software engineer with focus on Data Engineering, ETL processes, preferably with exposure to both batch and streaming data. The candidate should have familiarity with use of Databases and DataLake infrastructure and associated tools for ingestion, transformation and efficient querying across distributed data frameworks to include understanding of performance and scalability issues and query optimization.
Responsibilities:
- Design, build, and maintain workflows/pipelines to process continuous stream of data with experience in end-to-end design and build process of Near-Real-Time and Batch Data Pipelines.
- Expected to work closely with other data engineers and business intelligence engineers across teams to create data integrations and ETL pipelines to drive projects from initial concept to production deployment
- Maintaining and supporting incoming data feed into the data pipeline from multiple sources, including external customer feeds in CSV or XML file format to Publisher/Subscriber model automatic feeds.
- Active development of ETL processes using Python, PySpark, Spark or other highly parallel technologies, and implementing ETL/data pipelines
- Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers
- Ability to provide quick ingestion tools and corresponding access API’s for continuously changing data schema, working closely with Data Engineers around specific transformation and access needs
- Participate in L1 team rotation during business hours, in 1-week blocks.
Preferred:
- 1-2 year's experience developing applications with Relational Databases, preferably with experience in SQL Server and/or MySQL.
- Some exposure to database optimization techniques for speed, complexity, normalization etc.
Skills and Attributes:
1. Ability to have effective working relationships with all functional units of the organization
2. Excellent written, verbal and presentation skills
3. Excellent interpersonal skills
4. Ability to work as part of a cross-cultural team
5. Self-starter and Self-motivated
6. Ability to work without lots of supervision
7. Works under pressure and is able to manage competing priorities.
Technical qualifications and experience level:
1. 3-7 years in development using Java, Python, PySpark, Spark, Scala, and object-oriented approaches in designing, coding, testing, and debugging programs
2. Ability to create simple scripts and tools using Linux, Perl, Bash
3. Development of cloud-based, distributed applications
4. Understanding of clustering and cloud orchestration tools
5. Working knowledge of database standards and end-user applications
6. Working knowledge of data backup, recovery, security, integrity, and SQL
7. Familiarity with database design, documentation, and coding
8. Previous experience with DBA case tools (frontend/backend) and third-party tools
9. Understanding of distributed file systems and their optimal use in the commercial cloud (HDFS, S3, Google File System, Databricks)
10. Familiarity with programming languages API
11. Problem-solving skills and ability to think algorithmically
12. Working Knowledge on RDBMS/ORDBMS like MariaDb, Oracle and PostgreSQL
13. Knowledge of SDLC (Waterfall, Agile and Scrum)
14. BS degree in a computer discipline or relevant certification
*Key Requirements:
1) Strong on Python
2) Strong in the Linux environment (development in, navigating, and execution )
3) Some experience with Web UI development, using JavaScript frameworks, not simply JavaScript.
4) Understanding of web-based architecture
5) Ability to do some database relating development, both with use of code to read and update databases and also ability to create and manage DB tables.
...independence, and reach their full potential. Role Description This is a full-time on-site role for a Pediatric Occupational Therapy Assistant (COTA) at Olea Pediatric Therapy in Lakeville, MN. Olea Pediatric Therapy contracts with various ABA facilities in the Twin...
...Under the supervision of Regional Manager, the Staff Attorney will conduct the closings and supervise the settlement processes. The Staff Attorney will also assist with marketing efforts. Essential Responsibilities: Prepare and review deeds, contracts, powers of...
...First-Generation College Students ~ Ranked in the Top 100 High Schools in the Nation, U.S. News & World Report We serve the... ...and afterschool duty Growing professionally in your craft of teaching by both receive and giving feedback on a consistent basis from...
...success. Client Interaction Attend meetings and events to build strong client relationships. What Were Looking For: No Experience? No Problem! Enthusiasm for marketing, events, and brand promotions is key. Strong Communicator Confident in both written...
Required Skills & Experience Bachelors Degree in Geographic Information Systems or Geography is required Experience and proficiency with ESRI Suite products (ArcGIS Pro) Must enjoy the outdoors and be comfortable with some field work Nice to Have Skills &...