A Shiva Ramkrishna

Data Processing & Transformation:

• Processed large datasets from e-commerce, food delivery, and social media channels using Mage AI and PySpark, ensuring accurate mapping and transformation aligned with database schemas.
• Applied customized data cleaning, standardization, and validation techniques to maintain data quality, reliability, and consistency across domains, improving the accuracy of insights.

Data Quality Assurance & Analytics:

• Conducted rigorous data quality checks and validation to ensure accuracy, consistency, and reliability for analytics and reporting.
• Tracked key metrics to monitor data integrity and identified trends and anomalies to drive data-driven improvements.
• Supported analytics by setting up automated dashboards and reports, providing stakeholders with timely insights to guide decision-making.

Ad-Hoc Analytics & Use Cases:

• Tackled ad-hoc analytics tasks and custom use cases to support FMCG clients, including market trend analysis, customer segmentation, and distribution metrics of product subcategories.
• Created tailored visualizations to address specific client queries.
• Collaborated closely with client facing teams to identify data needs and translate insights into actionable business strategies, enhancing client satisfaction and retention.

Data Collection and Extraction:

• Developed and managed web scraping scripts for dynamic content extraction from major e-commerce sites (e.g., Amazon, Walmart, Lazada), ensuring efficient data extraction with techniques to handle pagination, proxies, and rotating user agents to avoid IP blocking.
• Collaborated with data engineers, analysts, and vendors to meet data requirements, retrieve data via APIs, and load it into databases. Built ETL pipelines using Airflow and Mage AI to streamline data flow for analytics.

Data Defining

• Worked for FMCG clients and performed feasibility studies in response to their requests to ensure alignment and clarity between the company and the client.
• Conducted extensive data discovery to identify sources to acquire diverse datasets relevant to specific project objectives.
• Utilized advanced search techniques and data mining tools to uncover hidden patterns and valuable insights within large datasets.
• Collaborated with cross-functional teams to define data requirements and ensure alignment with project goals.
• Developed and maintained a systematic approach for cataloging , mapping and organizing discovered data sources for easy retrieval on our database.
• Delivered practical recommendations to client-facing teams by leveraging insights obtained through data discovery, aiding in making well-informed business decisions

Data Collection

• Developed and implemented data scraping techniques to extract relevant information from various websites and online sources using libraries like BeautifulSoup, Selenium, Scrapy and Playwright.
• Collaborated with cross-functional teams to understand data requirements and customize scraping solutions accordingly.

Data Analysis

• Analyzed large datasets to identify trends, patterns and correlations for business insights.
• Conducted exploratory analysis of complex datasets using Python libraries such as NumPy and Pandas.
• Gathered requirements from stakeholders to facilitate report making for various use cases.
• Audited internal data and processes to identify and manage initiatives, improving business performance.
• Provided data-driven solutions to support decision making.

Data Visualization

• Designed and created compelling data visualizations to effectively communicate complex information and insights to the client facing teams.
• Conducted training sessions to train team members on effective data visualization principles and tools.

Data Accuracy

• Implemented robust data cleaning and preprocessing procedures to ensure accuracy and reliability of datasets.
• Conducted comprehensive data quality assessments to identify and address inconsistencies, errors, and missing values.
• Collaborated with data collection teams to establish standardized data entry practices, minimizing errors at the source.
• Developed and implemented data validation checks to identify and rectify anomalies during the cleaning process.
• Applied domain knowledge and contextual understanding to interpret and correct discrepancies in the data.
• Automated repetitive data validation tasks using python libraries like pandas and regex
• Conducted regular audits of cleaned datasets to maintain ongoing data quality and integrity.

• Conducted market research and data discovery to identify new data sources, trends, and opportunities.
• Gathered and analyzed data from multiple sources, utilizing tools like SQL, Excel, and Python to identify trends and actionable insights.
• Developed mappings for multiple lenses used in data tagging, ensuring accurate categorization and enhanced data retrieval across various datasets.
• Cleaned, validated, and transformed raw data into structured formats, ensuring data quality and consistency for analysis.
• Worked closely with domain experts and client-facing teams to understand data needs, deliver insights, and support data-driven decision-making.
• Applied SQL, Excel, and Python for data querying and analysis, and leveraged visualization tools to create impactful data presentations.