Candidates About

Kamal Kududhula

Hadoop Developer

·      Around eight years of extensive IT experience with multinational clients which includes 5 years of Hadoop related architecture experience developing big data / Hadoop applications.

·      Hands on experience with the Hadoop stack (MapReduce, HDFS, Sqoop, Pig, Hive, HBase, Flume, Oozie and Zookeeper)

·      Proven Expertise in performing analytics on Big Data using Map Reduce, Hive and Pig.

·      Experienced with performing real time analytics on NoSQL data bases like HBase.

·      Experienced with ETL to load data into Hadoop/NoSQL

·      Experienced with Dimensional modeling, Data migration, Data cleansing, Data profiling, and ETL Processes features for data warehouses 

·      Worked with Oozie work flow engine to schedule time based jobs to perform multiple actions.

·      Experienced in importing and exporting data between RDBMS and Tera Data into HDFS using Sqoop

·      Analyzed large amounts of data sets writing Pig scripts and Hive queries

·      Logical Implementation and interaction with HBase

·      Experienced in writing MapReduce programs &UDFs for both Hive & Pig in Java

·      Used Flume to channel data from different sources to HDFS.

·      Experience with configuration of Hadoop Ecosystem components: Hive, HBase, Pig, Sqoop, Mahout, Zookeeper and Flume.

·      Good experience in Hive partitioning, bucketing and perform different types of joins on Hive tables and implementing Hive SerDe like JSON and Avro.

·      Supported MapReduce Programs running on the cluster and wrote custom MapReduce Scripts for Data Processing in Java.

·      Developed Spark jobs using Scala in test environment for faster data processing and used Spark SQL for querying 

·      Good knowledge in Apache Crunch and Hadoop HDFS Admin Shell commands.

·      Experience with Testing Map Reduce programs using MRUnit and JUnit.

·      Experienced with implementing Web based, Enterprise level applications using J2EE frameworks.

·      Experience with web-based UI development using JQuery, CSS, HTML5, XHTML

·      Experienced with implementing/consumed SOAP Web Services using Spring CXF and Consumed Rest Web Services using Http Clients.

·      Experienced in writing functions, stored procedures, and triggers using PL/SQL.

·      Experienced with build tool ANT, Maven and continuous integrations like Jenkins.

·      Experienced in all facets of Software Development Life Cycle (Analysis, Design, Development, Testing and maintenance) using Waterfall and Agile methodologies

·      Motivated team player with excellent communication, interpersonal, analytical, problem solving skills and zeal to learn new technologies.

·      Highly adept at promptly and thoroughly mastering new technologies with a keen awareness of new industry developments and the evolution of next generation programming solutions.

·      Experienced working in BEA WebLogic Server and IBM WebSphere Application Server

·      Worked on various Tools and IDEs like Eclipse, IBM Rational, Visio, Apache Ant-Build Tool, MS-Office, PLSQL Developer, SQL*Plus

·      Conscientious team player and motivated to learn and apply new concepts. Always aspires to exceed client expectations and to effectively collaborate with several cross-functional teams. 

 

 

Technical Skills:

Hadoop Ecosystem	Hadoop, MapReduce, Spark, Sqoop, Hive, Oozie, PIG,  HDFS, Zookeeper, Flume
Languages	C, C++, JAVA, Python, PHP,  SQL, PL/SQL, PIG Latin, HiveQL, Unix shell scripting
Java & J2EE Technologies	Core Java, Servlets, JSP, JDBC, JNDI, Java Beans
Frameworks	MVC, JMS, JUnit, MRUnit , JTest, LDAP
No SQL Databases	HBase, Cassandra, Mongo DB
IDE's & Utilities	Eclipse, NetBeans
Databases	Oracle, My SQL, DB2, MS SQL Server, PostgreSQL
Operating Systems	Linux, MacOS, Windows 8, Windows 7, Vista, XP, Windows 95/2000 and MS-DOS
Web Services	WSDL, SOAP, REST, Jersey

 

EDUCATION:

Bachelor of Technology

Work Experience

 

Responsibilities

 

•   Built python script to extract the data from the Hawq tables and generated a “dat” file for the downstream application

•   Tested the reports generated off of data migrated to Hadoop data lake.

•   Built a generic framework to parse raw data with fixed length using python which takes JSON Layout for the fixed positions of the strings and load the data into Hawq tables.

•   Built generic framework that transforms two or more data sets in HDFS using python.

•   Built generic frameworks for Sqoop/Hawq to load data from SQL server to HDFS and HDFS to Hawq using python.

•   Performed extensive data validation using Hawq partitions for efficient data access.

·      Optimizing the Hive queries using Partitioning and Bucketing techniques, for controlling the data distribution.

·      Written custom UDF’s in java and implemented through pig.

•   Built generic framework that allows for us to update the data in a Hawq tables using python.

•   Coordinated in all testing phases and worked closely with Performance testing team to create a baseline for the new application.

•   Created automated workflows that schedule jobs daily for loading data and other transformation jobs in using cisco tidal.

•   Created PostgreSQL functions (stored procs) to populate the data into the tables on daily basis.

•   Developed functions using PL python for various use cases.

•   Had to do the Cluster co-ordination services through ZooKeeper.

•   Troubleshooting and monitoring Hadoop services using Ambari.

•   Documented technical design documents and production support documents.

•   Worked on SSIS and SSRS tools to aid in decommission of the data from SQL to distributed environment.

•   Wrote python scripts to create automated workflows.

•   Built python process to perform regression testing of the data based on the mapping documents provided, after migrating it to the data lake.

•   Build test cases using HP ALM tool to perform integration testing of the migrated data.

 

Environment: Pivotal HD 2.0, HDFS, HAWQ, Sqoop, Hive, Pig, SQL, Zookeeper, PostgreSQL, PgAdmin, Cisco tidal, shell scripting, Python, Java.

 

 

Responsibilities:

 

·      Handled importing of data from various data sources, performed transformations using PIG and Hive to load data into HDFS

·      Used Flume and Sqoop to load data from multiple sources into HDFS

·      Configured Zookeeper to restart the failed jobs without human intervention.

·      Written Sqoop incremental import job to move new / updated info from Database to HDFS.

·      Experience in joining raw data with the reference data using Pig scripting.

·      Visualize the HDFS data to customer using BI tool with the help of Hive ODBC Driver.

·      Analyzed the data by performing data profiling

·      Created Oozie workflow engine to run multiple Hive and Pig jobs.

·      Created Oozie coordinated workflow to execute Sqoop incremental job daily. 

·      Coordinated with ETL engineers for ingestion of various data sources

·      Designed, developed and tested application

·      Analyzing the data for defects and fixing the defects.

·      Suggested effective enhancements for existing automation framework.

 

Environment: Hadoop, HDFS, Pig, Hive, HDFS, Zookeeper, Sqoop, Flume, Oozie and HUE.

 

 

Responsibilities: 

·      Worked on importing data from various sources and performed transformations using MapReduce, Hive to load data into HDFS

·      Responsible for building scalable distributed data solutions using Hadoop 
Written various Hive and Pig scripts

·      Created HBase tables to store variable data formats coming from different portfolios

·      Configured multiple remote akka worker nodes and Master nodes from scratch to as per the software requirement specifications.

·      Performed real time analytics on HBase using Java API and Rest API.

·      Participated in major phases of software development cycle with requirement gathering, Unit testing, development, and analysis and design phases using Agile/SCRUM methodologies.

·      Worked with BI teams in generating the reports and designing ETL workflows on Tableau.

·      Implemented HBase Co-processors to notify Support team when inserting data into HBase Tables.

·      Configured Sqoop jobs to import data from RDBMS into HDFS using Oozie workflows.

·      Worked on setting up Pig, Hive and HBase on multiple nodes and developed using Pig, Hive, HBase and MapReduce

·      Worked on compression mechanisms to optimize MapReduce Jobs

·      Analyzed the customer behavior by performing click stream analysis and to ingest the data used flume.

·      Experienced with working on Avro Data files using Avro Serialization system.

·      Solved small file problem using Sequence files processing in Map Reduce.

·      Implemented business logic by writing UDF's in Java and used various UDF's from Piggybanks and other sources.

·      Wrote various Spark transformations using Scala to perform various data cleansing, validation, transformation and summarization activities on user behavioral data.

·      Continuous monitoring and managing the Hadoop cluster using Cloudera Manager 
Worked on Oozie workflow to run multiple jobs.

·      Exported the analyzed data to the relational databases using Sqoop for visualization and to generate      reports for the BI team. 

Environment: Horton works, Map Reduce, Spark, HBase, HDFS, Hive, Pig, Java, SQL, Cloudera Manager, Sqoop, Flume, Oozie, Java (jdk 1.6), Eclipse

 

 

 

Responsibilities: 

·      Developing light weight business component and integrated applications using struts

·      Designing and developing front-end, middleware and back-end applications.

·      Optimizing server/client side validation.

·      Transfer old Perl scripts into new Python scripts, add new functions and features. Develop automated test method and documentations for these scripts.

·      Worked together with the team in helping transition from Oracle to DB2.

·      Developed the global logging module which was used across all the modules using Log4Jcomponents.

·      Developed the presentation layer for the credit enhancement module in JSP.

·      Struts were used to implement the Model View Layer (MVC) architecture. Validations were done on the client side as well as the server side.

·      Involved in the configuration management using ClearCase.

·      Detecting and resolving errors/defects in the quality control environment.

·      Using Ibatis for mapping Java classes with database.

·      Involved in Code review and integration testing.

·      Used Debugging tools such as PMD, Find Bugs and checkstyle. 

Environment: Java v1.6, J2EE 6, Struts 1.2, iBatis, XML, JSP, CSS, Python, HTML, JAVASCRIPT, JQuery, Oracle 10g, DB2, Unix, RAD, ClearCase, WebSphere V8.0 (beta)