The Data Engineer will be responsible for building, deploying and maintaining data models, data pipelines for batch and real-time data analytics. This position will work on the collecting, storing, processing, and analyzing of data sets from 20+ disparate data sources. The Data Engineer must have advanced analytical skills, showing fluency in the use of platforms and tools such as MS Azure/AWS and Python, Spark, Scala, Java and T-SQL. The role also requires one to be technologically adept, demonstrating strong computer science skills. The Data Engineer must additionally be capable of developing databases using SSIS packages, pipelines orchestration, T-SQL, MSSQL and Spark scripts.
This role is responsible for creating data solutions for self-service tools across the organization within the boundaries of our data policy, standards and controls. This role will also work closely with the business stakeholders and technical teams and provide expertise in designing and building the analytics platform. The ideal Data Engineer will have both a data engineering background and a strong business acumen that enables them to be strategic and add value to the Pethealth business and its customers. This role will participate in a wide range of problem-solving discussion, requiring extensive use of data collection, data exploration, and analysis techniques.
- Participate in building the Data and Analytics platform
- Data ingestion and orchestration: ingest (internal, external, third party structured and unstructured data), transform, curate and publish data sets for Data Science, AI applications infrastructure and self-service solutions
- Data inventory: while building analytics-ready data, create a catalog of critical data elements and data dictionaries
- Data Quality: perform data profiling of critical data elements, providing trending and insights that drive action to improve data quality, performance and cost
- Data Integration: integrate a variety of data sources including third party data and ensure successful application within current environments
- Data Governance: implement a comprehensive end to end data governance practice
- Monitoring: support a monitoring system to ensure the timeliness and accuracy of the data in the analytics platform
- Collaboration: implement solutions required by Data Scientists and Analysts to enable advanced analytics techniques such as Machine Learning and AI modeling to improve customer experience, customer engagement and company business processes
- Other duties as assigned
- Bachelor degree required in Computer Science, Engineering or related field /or equivalent experience
- 3+ years of working experience with SQL server and coding with T-SQL
- 2+ years of experience in data pipeline developing and maintaining a cloud data architecture AWS/Azure/Google Cloud and developing and maintaining data sets used for Data Science and AI applications.
- 2+ years of experience working with cloud analytics platforms in MS Azure or AWS (Databricks, Synapse, etc)
- 2+ years of engineering scalable software using big data technologies (Hadoop, Hive, Spark, etc.)
- 2+ years completing tasks related to the data infrastructure required for optimal extraction, transformation and loading of data using data pipelines in MS SSIS, Azure Data Factory, Informatica or StreamSets
- 1+ year of experience in cloud data warehousing
Required Qualifications (Certifications, Licenses, Software Programs, KSA’s): Stack (required in bold):
- Cloud Technologies: Azure, Databricks, Delta lake, AWS S3
- Data pipelines/ETL: Azure Data Factory, Databricks, StreamSets, MS SSIS, Informatica
- Data warehousing: Azure Synapse, Oracle 12c, Databricks
- Relational Databases: SQL Server all versions since 2000, Oracle 11g, 12c
- Big Data technologies: Spark
- Languages: C++, C#.Net, VB.net, Java, Python, R
- Code Versioning Tools: VSS, SVN, TFS, Git
- Issue tracking tools: Jira, Azure DevOps
- Data Visualization: PowerBI, OBIEE, Microstrategy, Tableau, Looker
- Tools and IDE: VS, Pycharm, IntelliJ, DataGrid, Eclipse, SSMS, Azure Data Studio, Oracle SQL Developer
- Advanced understanding of metadata management, data modeling, data catalog
- Experience in the financial services/Insurance industry is an asset
- Advanced analytical and problem-solving ability
- Excellent communication skills both in written and verbal format
- Driven to move fast to deliver business values
- Attention to detail and a commitment to quality
- Experience working in an agile environment
- Frequent communications, verbal and written.
- Must be able to remain in a stationary position 80% of the time.
- Constantly operates a computer and other office productivity machinery, such as a phone with headset, calculator, copy machine, and computer printer.
If selected for an interview, you will be contacted by email within 7 business days.
Pethealth is an equal opportunity employer and does not discriminate against any employee or applicant for employment because of age, race, religion, disability, gender, sexual orientation or national origin. Accommodations are available upon request for candidates taking part in all aspects of the selection process.
- Job Family Services - IT Development
- Pay Type Salary
- Travel Required No
- Required Education Bachelor’s Degree
- Toronto, ON, Canada