Sagnik Bhattacharya

Driven Brands-Data Engineer Consultant

October 2020-Current

• Migrated the existing data from Mainframe/Teradata/SQL server to Hadoop & perform ETL operations on it.
• Designed & implemented SQOOP incremental imports,delta imports on tables without primary keys & dates from teradata & append directly into Hive warehouse.
• Worked with Avro & Parquet file formats & used various compression techniques to leverage the storage in HDFS.
• Designed & developed ETL workflows using Oozie & automated them using autosys.
• Reduced the latency of Spark jobs by tweaking spark configuration & following other performance & optimization techniques.
• Responsible for loading unstructured & semistructured data into Hive by creating static & dynamic partitions.

AXA Insurance-Data Engineer Associate Consultant

July 2019-September 2020

• Using various transformations & actions for cleansing the input data.
• Implemented Spark using Scala and Spark SQL for faster testing and processing of data.
• Used Impala for querying HDFS data to achieve better performance.
• Parsing JSON data using Spark.
• Used Spark API over Hortonworks Hadoop YARN to perform analytics on data in Hive.
• Worked in close association with the Business Analysts and DBAs for gathering requirements, business analysis, and testing and project coordination and participated in data modeling JAD sessions.

Franklin Templeton-Data Engineer Associate consultant

January 2019-July 2019

• Used SQOOP to import/export various RDBMS(Teradata/Oracle) to Hadoop cluster & vice versa.
• Used airflow to build ETL solutions.
• Developed & maintained data pipeline, ingesting data across different sources using Redshift,S3 & Python.

Morse Corp-Mainframe Database administrator

January 2012-January 2019

• Executed effective processes for database backup,recovery and maintenance utilities.
• Executed effective processes for development of SQL scripts,stored procedures & application functions.
• Suggested enhancements for DB2 applications in terms of database performance & resource usage.
• Assisted in measurement of database performance,capacity monitoring & tuning processes.
• Provided technical assistance for development of database security,testing and control procedures.

EDB Ergo-Mainframe Developer

• Mainframe application abend resolution.
• Implement and support applications in a mainframe production environment
• Used JCL,COBOL,VSAM,DB2,IMS,Expeditor.
• Suggested technical alternatives for streamlines processes & systems.
• Proficient in CA-7 scheduling tool.
• Experience in developing/maintaining mainframe based management systems using COBOL,SQL,CICS

Wellpoint Anthem

• Provided application software development services or technical support in a defined project.
• Provided software application support. Included day call & night call responsibilities on a rotating schedule.
• Provided design development of projects with COBOL/CICS and mainframe development, migrations & additions to existing applications.
• Mainframe application abend resolution.
• Working knowledge and understanding of DB2,SQL,JCL,CICS.
• Created process flows & functional documentation.
• Running a SPUFI query to update the DB2 tables with the participating applications list & details.
• Updating Incident & problem tickets,Test information documents,abend logs & defect tracker documents on a regular basis.
• Developed & maintained the mainframe system, supporting various COBOL programs,JCL PROCS,Copy members & Job streams.