Hrithik Kari

Data Engineering Analyst

Accenture

12.2022 - Current

Developed Scalable Data Pipelines: Played an integral role in designing and developing data pipelines for a client project, Value Insights Platform (VIP), leveraging financial data from 100K+ public and private companies.
Data Pre-processing: Pre-processed terabytes of incoming vendor data in MS SQL Server using custom Stored Procedures, Views, and Materialized Tables for optimized querying and analytics.
Cloud Integration: Imported Materialized Tables into AWS S3 and AWS Athena using VPC connections, leveraging AWS Glue Crawlers to read and manage schemas dynamically.
Large-Scale Data Processing: Processed terabytes of data using AWS Glue on the pySpark framework, combined with AWS Lambda functions for transformation and computation.
Data Orchestration and Automation: Orchestrated complex data workflows using AWS Step Functions, Glue Workflows, and Apache Airflow, ensuring robust automation and error handling.
Data Storage and Backend Integration: Stored processed data in DynamoDB (NoSQL) and Aurora PostgreSQL in the form of relational tables and materialized views. Designed and deployed RESTful APIs with Amazon API Gateway, enabling seamless interaction with the PostgreSQL backend using Python-based Lambda functions for CRUD operations.
Event-Driven Architecture: Implemented event-driven automation using AWS EventBridge to trigger workflows and Lambdas, ensuring real-time data processing.
Prompt Engineering for LLMs: Developed advanced prompt engineering techniques to enable LLMs to generate SQL queries, streamlining backend data management.
API Testing: Hands-on experience with Postman for API testing and validation.
End-to-End Visualization: Enabled insights visualization by preparing and integrating backend data into web applications for business intelligence and analytics.