Kiran Reddy Mareddy

Employment: Having 7.8 years of experience in the SDLC including application design, development, and production support and maintenance projects. Certified as a Hadoop Developer.

Senior Big Data Engineer

Cognizant Technology Solutions (CTS)

Dec 2014 – Till Date

• Working on Agile iteration methodology format for Project PDLC

• Working for 'Naturalistic Driving Data Study' with DataOps and Customer teams to formulate specific Data         Lake design (for   Data Ingestion, Migration, and Analytical consumption needs)

• Preparing the workflow design flow chat diagram, get approval from Client core team

• Written "Kill and fill", and "incremental" Sqoop jobs to import data from RDBMS (SQL Server) to Hive Staging tables

• Designs and implementing Scala programs using Spark Dataframes and RDDs for transformations and actions on input data

• Improved the Hive queries performance by implementing partitioning and clustering and Optimized file formats (ORC)

• Scheduled the workflows using Oozie workflow scheduler

• Worked on GDPR (General Data Protection Regulation) to remove the Europe user data

• Analyzed the Data model and identified the GDPR columns, which matches with my current system to upstream system

• Converted existing non-ACID (External) tables to ACID (Managed) tables using Buckets and clustered by columns

• Created the Data migration scripts, to migrate from old non-ACID tables to ACID tables

• Removed the EU data from tables using 'GDPR Framework'

Big Data Engineer

Larsen & Toubro (L & T) Infotech

May 2011 – Dec 2014

• Validated and transformed the input text file using Java Decoder, loading the validated and transformed data into Hive   staging table

• Identified the duplicate rows and insert into Master Duplicate tables, and segregate the data with respect to the data   security level say SPII, PII and Non-PII, and load to Master tables with extra columns (audit columns)

• Created the Views with respect to region, to give access to different groups

• Apply the data Security using Ranger Policies, applying Ranger policies for Hive tables based on Security level (SPII,   PII & Non-PII), and for Views also

• Migration DB2 Historical data to Hadoop (Hive) using Sqoop

• Analyzed the DB2 data import requirements, and import using Sqoop to Hive staging tables,

• Prepared the data flow design document and preparing the Sqoop jobs for importing DB2 data to hive

• Masked the secured data, inserting the staging data to various Master tables like (Vendor, Products, Warehouses, Price   details, Sales, Invoice tables etc.)

• Migrated the DB2 reporting sql scripts to Hive ql’s for reporting system, and loads the data into Hive reporting table   for analysis

• Prepared the Oozie jobs to schedule the Sqoop, Hive scripts to load the historical data

• Prepared the test data, test cases document, and execute the test cases in DEV environment

• Moved the tested code to QA with help of SCM team, giving function KT to QA testing team, and support QA  and UAT

• Prepared and presented the Production checklist to AM team, build and run the workflow in Production with help of AM team

Software Engineer

Artech Information Systems

Aug 2010 – May 2011

• Translated the requirements from the business and analyst into technical code

• Using connection managers connects with Excel worksheet, delimited or fixed width flat files

• Build Control Flows, making connections between the objects like Tasks, Maintenance Plan tasks, The Data    Flow Task and   Containers

• Created Complex ETL Packages using SSIS to extract data from staging tables to partitioned tables with incremental load

• Build Data Flows objects, mapping the source of the data using 'Data Flow Sources', alter and manipulate the data in   various ways using 'Data Flow Transformations'

• Finally send the transformed data to 'SQL Server' using 'Data Flow Destinations'

• Prepared the test data, test cases document, and executed the test cases in DEV environment

• Provided support and fix issues in the packages

ü  MCA (Master of Computer Applications), from Sri Venkateswara University, India

ü  B.Sc Computers (Bachelor of Computer Science), from Sri Venkateswara University, India

ü  Certified as a Hadoop Developer

ü  Received “Best performer of the year” certificate from the Client

Technical Experience

·         Working on Data ingestion with various data sources like RDBMS, text files to Hadoop, designs and implementing the Scala programs using Spark Dataframes and RDDs for transformations and actions on input data, and loading the structured data into Hive tables using Spark DataFrame

·         Built high performance data integration solutions using SSIS ETL tool (including extraction, Transformation, and load (ETL) packages for data warehousing)

·         Migrating DB2 tables to Hadoop (Hive) using Sqoop (migrating DB2 reporting sql’s to Hive ql’s for reporting system)

·         Worked on GDPR to remove the EU user data, which has Opt out from the system. (Converting existing non-ACID tables to ACID tables, migrating old data to new ACID tables, Removing the EU data from tables using 'GDPR Framework')

Technologies and Languages