Job Description
Expert level understanding of distributed computing principles(Big Data Processing). Expert level understanding and hands on experience in designing Data Model. Hands on experience in designing Data Pipelines for ETL process(Databricks/Delta Live table/Synapse/Data Factory). Experience with designing and setting up Delta lakehouse architecture( Data vault 2.0, Data mart/Star Schema, Snowflake) Experience in applying best practises in big data sets(Query Optimization, Data Partitioning, Relative Filters) Good knowledge and hands on experience in Apache Spark(Batch and Streaming data). Hands on experience in programming with Python and SQL and maintain code quality/test coverage. Data Engineering Methodology like (SCD, complex analytical queries with huge amounts of data). Practitioner of AGILE methodology(Scrum/Kanban) Knowledge and experience in Code Management, Code Versioning, Git flow, Release Planning. Excellent communication, presentation ,documentation skills. Mindset – taking initiatives, team player, keen to learn, adapt changes.
Responsibilities
Good to have Experience in setting up Devops pipeline on Kubernetes. Working experience as Data Engineer in Cloud environment (Microsoft Azure). Experience with integration of data from multiple data sources(Telnet/MQTT/HTTP/Pub-Sub). Work experience with streaming technologies like Apache Flink, Apache Storm. Experience with querying technologies like Trino, DremIO. Hands on experience with BI tool(Power BI, Tablue)