BALAJI NAIDU
System Integration Specialist
Chittoor
Summary
To achieve excellence in technology and use my skills to the utmost benefit of the organization, and to acquire a technical position where engineering experience will significantly contribute to product and process development, increase in productivity and better market position.
Experience: I carry around 9 years of experience in Big data, ETL Data warehousing and BI Testing. Confident Hadoop Developer with 5 years of experience and phenomenal technical expertise. Successfully working with Spark, Scala, Hive, AWS Redshift, Hue, HDFS, Hive, Sqoop, EMR, Oozie and Yarn include designing, developing applications using Hadoop Ecosystem components.
Technical skills: One of the key person in the team potentially play a role in Hadoop framework Development and Support. An Individual with project planning, task delegation and scheduling expertise.
Strong technical skills in Hadoop, Spark, Scala, AWS Redshift ,Hadoop development, Hue, DevOps (HDFS, Hive, Sqoop, Oozie and Ambari include designing, developing applications) using Hadoop Ecosystem components.
- Having around 9 years of overall IT experience Hands on 5 years of experience in Bigdata and Hadoop ecosystems like HDFS, YARN, Sqoop, Hive and Oozie,. Having 4+ year's experience in Spark development and support using Scala and Spark Sql.
- Load and transform large sets of structured, semi-structured data from Relational Database Systems to HDFS and vice-versa using Sqoop tool.
- Having good knowledge in Python and Pandas.
- Data ingestion and refresh from RDBMS to HDFS using Apache Sqoop Designing and creating Hive external tables, using shared meta-store instead of Derby, and creating Partitions and Bucketing techniques in Hive using Hive Query Language (HQL).
- Proficient in using RDBMS concepts, whereas excellent understating in Redshift. Excellent communication, organizational skills, attention to detail & the ability to meet tight deadlines.
- Exposure on complete Informatica Development activities. Data Warehousing Implementation by using ETL tool Informatica Power Centre 9.6, 9.1.
- Having experience of Data Warehousing life cycle -ETL Process. Extensive experience in Extraction, Transformation and Loading of data directly from different heterogeneous source systems. Skill Inventory.
- Have experience in using DevOps Tools
Overview
14
14
years of professional experience
4
4
years of post-secondary education
Work History
System Integration Specialist
NTT DATA
Bangalore
04.2018 - Current
- Project : Hilton Hospitality
- Client : Hilton
- Tools : HDFS,Spark,Scala, Python, Hive, Oozie,Yarn, Redshift, AWS s3, and EMR
- Environment : Putty, Linux, Windows
Hadoop Developer
05.2017 - Current
- Hilton is one of the largest and fastest growing hospitality companies in the world, with more than 5,200 properties with more than 856,000 rooms in 105 countries and territories
- In the nearly 100 years since our founding, we have defined the hospitality industry and established a portfolio of 14 world-class brands, including our flagship Hilton Hotels & Resorts brand, which is the most recognized hotel brand in the world
- Our premier brand portfolio also includes Waldorf Astoria Hotels & Resorts, Conrad Hotels & Resorts, Canopy by Hilton, Curio Collection by Hilton, Tapestry Collection by Hilton, Doubletree by Hilton, Embassy Suites by Hilton, Hilton Garden Inn, Hampton by Hilton, Tru by Hilton, Homewood Suites by Hilton, Home2 Suites by Hilton and Hilton Grand Vacations
- We have more than 71 million members in our award–winning customer loyalty program, Hilton Honours
- Roles and Responsibilities:
- Worked with HDP Cluster containing 64 Nodes Production Cluster and 36 Nodes Development cluster
- Responsible for coordinating end to end project related activities
- Involved in all phases of the SDLC including analysis, design, development and testing of the application in the Hadoop cluster
- Developed Shell Scripting to pull the files from informatica server to Hadoop
- Responsible for creating hive external & internal(orc) tables
- Developed Spark Ingestion Framework to load the data from Hive External Tables to internal tables at one shot
- Understanding the informatica mappings and documented the business logic
- Developed Spark code for consumption layer which includes informatica logic and further loaded data into hive fact & dimension tables
- Created Unit Test cases and executed the same
- Informatica 9.6 & HDFS, Hive, Ozie, Ambari, Scala and Spark
- Database : Oracle.
Senior Software Engineer
Dell International Services(EXPERIS IT Pvt Ltd)
Bangalore
09.2016 - 03.2018
- Worked with project managers, developers, quality assurance and customers to resolve technical issues.
- Trained and mentored junior developers and engineers, teaching skills in (Informatica, ETL Testing) and working to improve overall team performance.
- Worked with software development and testing team members to design and develop robust solutions to meet client requirements for functionality, scalability and performance.
- Reviewed project specifications and designed technology solutions that met or exceeded performance expectations.
GM
09.2016 - 04.2017
- Project Description, Financial is the captive finance company and a wholly owned subsidiary of General Motors Company and is headquartered in Fort Worth; Texas.GM Financial is a global provider of automobile finance solutions, with operations in the U.S., Canada, Europe and Latin America
- Through there long-standing relationships with auto dealers, offer attractive retail loan and lease programs to meet the
- Unique needs of each customer and also offer commercial lending products to dealers to help them finance and grow their businesses
- GM Financial having operations in 19 countries, providing auto finance solutions to approximately 16,000 dealers worldwide
- Roles and Responsibilities:
- Responsible for importing data from Mysql to HDFS or Hive using Sqoop tool
- Created and worked Sqoop jobs with incremental load to populate Hive External tables
- Responsible for creating Hive external tables, loading data and writing hive queries
- Responsible for creating Hive external tables, loading data and writing hive queries in SparkSql
- Used Partitioning and Bucketing techniques in hive to improve the performance
- Involved in choosing different file formats like RC, ORC over Textile format
- Potentially choose the compression techniques for intermediate data and output data
- Potentially perform the hive query Performance tuning
- EDW_ Metrics
- Client : Trade Mark Electronics
- Environment : Informatica, UNIX and SQL developer
Software Engineer
SLK Software Services(ISOMETRIC Techonology Services)
Bangalore
12.2013 - 08.2016
- Trademark Electronics is Electronic equipments Manufacturing Company
- They have products including a wide range like household appliances, computer peripherals, entertainment, medical electronics etc
- The main objective of the project was assisting the senior and middle level management of the organization to improve the sales and get the knowledge of the new business opportunities
- Roles and Responsibility:
- Extensively used Informatica data quality to design multiple mappings based on business logic provided in the Design level document
- Involved in code improving methods like Data cleansing and Data merging
- Worked with various developer modules like profiling, standardization and matching
- Developed ETL mappings, transformations using Informatica Power Center
- Developed mappings using various transformations like update strategy, lookup, stored procedure, Router, Filter, sequence generator, joiner, aggregate transformation and expression
- Involved into source code review, the mappings created by my team members
- Prepared deployment document and assisted to deployment team while migration
- Involved in creating test cases
- Involved in performance tuning by optimizing the sources, targets, mappings and sessions
- Mappings were designed to cater the slowly changing dimensions
- Developed several reusable transformations and mapplets that were used in other mappings.
Education
Bachelor of Technology - ECE
JNTU Anantapur
01.2009 - 01.2013
Skills
Informatica Power Centre 9X
HDFS, Spark, Sqoop, Hive, Oozie, IntelliJ, Yarn
Databases : Oracle, HIVE, Red shift & Sql Server
Languages : Scala & PythonOperating Systems : Windows, UNIXDevOps tools and Git
DB Testing, ETL Testing and BI Testing
Timeline
System Integration Specialist
NTT DATA
04.2018 - Current
Hadoop Developer
05.2017 - Current
Senior Software Engineer
Dell International Services(EXPERIS IT Pvt Ltd)
09.2016 - 03.2018
GM
09.2016 - 04.2017
Software Engineer
SLK Software Services(ISOMETRIC Techonology Services)
12.2013 - 08.2016
Bachelor of Technology - ECE
JNTU Anantapur
01.2009 - 01.2013