Candidates About

 Duvvur

Professional Summary:

·         7 Years of experience in IT which includes 4 years of experience in Big Data ecosystem related technologies.

·         Proficient experience in Installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster.

·         Hands on exposure on major components in Hadoop Ecosystem like Hadoop Map Reduce, HDFS, HIVE, PIG, Pentaho, HBase, Zookeeper, Sqoop, Oozie, Cassandra, Flume and Avro.

·         Good experience with Oozie Workflow Engine in running workflow jobs with actions that run Hadoop Map/Reduce and Pig jobs.

·         Experience in understanding the security requirements for Hadoop and integrate with Kerberos authentication and authorization infrastructure.

·         Expert in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe.

·         Good experience in Apache Flume for efficiently collecting, aggregating, and moving large amounts of log data.

·         Experience and in-depth understanding of analyzing data using HIVEQL, PIG Latin.

·         Experience in managing Hadoop clusters using Cloudera Manager Tool.

·         Hands on experience in Sequence files, Combiners, Counters, Dynamic Partitions, Bucketing.

·         Hands-on experience in performing analytics using Apache Spark.

·         Good Knowledge on Map Reduce design patterns.

·         Responsible for setting up processes for Hadoop based application design and implementation.

·         Developed GUI applications by using HTML, CSS, AJAX, JSP and JavaScript libraries to implement user interface screens.

·         Experience in Core Java technologies (JDBC, Multi-Threading, Networking, Generics, Oops concepts, collections, exception handling).

·         Strong experience in Design, Development, Testing, Deploying of Object Oriented, and Web based Enterprise Applications using Java/J2EE technologies.

·         Experienced in developing enterprise applications using MVC frameworks such as Struts, Spring MVC.

·         Proficiency in multiple databases like MongoDB, Cassandra, MYSQL, Oracle 9i, 10g, 11g and MS SQL Server. 

·         Experienced in developing applications using application/Web servers such as Web Sphere, Web logic and Tomcat.

·         Experienced by using Hibernate interfaces, annotations, JDBC and SQL in implementing DAO layers.

·         Extensive knowledge of Agile and Scrum methodologies to develop best practices for software development and implementation.

·         Worked and learned a great deal from (AWS) Cloud services like EC2, S3, EBS, RDS and VPC. 

·         Worked on various Tools and IDEs like Eclipse, IBM Rational, Apache Ant-Build Tool, MS-Office, PL SQL Developer, SQL*Plus.

·         Expertise in using Java API and Sqoop to export data into DataStax Cassandra cluster from RDBMS. 

 

 

 

 

Technical Skills:

Hadoop Ecosystem	HDFS, Map Reduce Hive, Pig, Pentaho, HBase, Storm, Zookeeper, Sqoop, Oozie, Flume, Avro and YARN
Web Technologies	Core Java, J2EE, Servlet, JSP, JDBC, XML, AJAX, SOAP, WSDL, REST.
Methodologies	Agile, UML, Water-Fall and Test Driven
Frameworks	Hibernate 2.x/3.x, Spring 2.x/3.x,Struts 1.x/2.x and JPA
Programming Languages	Java, Unix Shell scripting, HTML, C, C++,SQL, PL/SQL
Data Bases	Oracle 11g, DB2, MS-SQL Server, MYSQL, MS-Access, Mongo DB, Cassandra.
Application Servers	Web Logic, Web Sphere, Apache Tomcat.
Web Design Tools	HTML, DHTML, AJAX, JavaScript, JQuery and CSS, Angular Js, Ext.JS and JSON
Operating Systems	Windows, Linux, Unix
Development / Build Tools	TOAD, SQL Developer, SOAP UI, Eclipse, Ant, Maven, IntelliJ, JUNIT and log4J.
ETL Tools	Talend, Informatica, Pentaho

 

 

Professional Experience:

 

Client: AT&T, Palo Alto, CA                                                                        Sep 2016 – Present  

Role: Hadoop Developer

 

Responsibilities:

·         Installed and configured Hadoop, Map Reduce, HDFS, developed multiple maps reduce jobs in Java for data cleaning and pre-processing. 

·         Involved in creating Hive tables, loading the data and writing hive queries that will run internally in a Map Reduce way.

·         Designed and implemented Map Reduce based large-scale parallel relation-learning system. 

·         Analysed Hadoop cluster and Big Data analytic tools including Pig, Hive, HBase and SQOOP. 

·         Developed Spark scripts by using Scala Shell commands as per the requirement.

·         Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs and Scala.

·         Used Spark Streaming to divide streaming data into batches as an input to Spark engine for batch processing.

·         Handled data from RDBMS and streaming sources used Apache Spark for data processing.

·         Responsible for developing Kafka as per the software requirement specifications.

·         Involved in monitoring data and filtering data for high speed data handling using Kafka. 

·         Worked in Spark streaming to get ongoing information from the Kafka and store the stream information to HDFS.

·         Implemented UNIX shell scripts to perform cluster admin operations and used Zookeeper for Cluster coordination services. 

·         Written Storm topology to accept the events from Kafka producer and emit into Cassandra DB.

·         Imported data from various resources to the Cassandra cluster using Java APIs.

·         Configured, Documented and Demonstrated inter node communication between Cassandra nodes and client using SSL encryption. 

·         Worked on Cassandra in creating tables to load large sets of semi structured data coming from various sources.

Analyzed the data to export Cassandra using Sqoop and to generate reports for the BI team.

Ingested the data from Relational Databases to HDFS using SQOOP.

·         Configured the servers for Auto scaling and elastic load balancing in AWS EC2 and used AWS services like EC2 and S3 for small data sets. 

·         Developed Pig Latin scripts for replacing the existing legacy process to the Hadoop and the data is fed to AWS S3.

·         Strong experience in working with ELASTIC MAPREDUCE and setting up environments on Amazon AWS EC2 instances.

·         Created S3 buckets and also managing policies for S3 buckets and Utilized S3 bucket and Glacier for storage and backup on AWS. Developer Columnar Data warehouse using the Red shift for analyzing the 4TB of customer data. 

·         Used Pig as ETL tool to do joins and some pre-aggregations before storing the data onto HDFS. 

·         Analyzed the data by performing Hive queries and running Pig scripts as per customer requirements.

·         Used Spark SQL to load tables into HDFS to run select queries on top.

·         Involved in loading data from UNIX file system to HDFS. 

·         Experienced in handling Avro data files in Map Reduce programs using Avro data serialization system.

 

 Environment: HDFS, Hadoop, AWS, Elastic Map Reduce, Spark, Pig, HBase, SQL, UNIX, Hive, Sqoop, Kafka, Spark Streaming ,Linux shell scripting, ETL, Zoo Keeper, Eclipse and Cassandra.

 

Client: Verizon, New Jersey, NY                                                                    Apr 2015 – Aug 2016

Role: Hadoop Developer

 

Responsibilities:

·         Involved in HBase Java API to populate operational HBase table with Key value.

·         Installed and configured Hive, Pig and Sqoop on the HDP 2.0 cluster. 

·         Performed data analysis with HBase using Hive External tables.

·         Used Map Reduce to move bulk amount of data into HBase to Integrate with NoSQL database like HBase.

·         Performed real time analytics on HBase using Java API and Fetched data to/from HBase by writing Map Reduce job.

·         Installed and configured Hadoop Map Reduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and Processing. 

·         Developed Map-Reduce jobs on Yarn and Hadoop clusters to produce daily and monthly reports.

·         Supported Map Reduce Programs those are running on the cluster.

·         Implemented intermediate functionalities like events or records count from the Flume sinks.

·         Designed workflows by scheduling Hive processes for Log file data, which is streamed into HDFS using Flume.

·         Imported data using Sqoop to load data from MySQL to HDFS on regular basis. 

·         Extracted the data from MySQL into HDFS using Sqoop. 

·         Used Sqoop to load the CDRs from relational DB and other sources to Hadoop cluster by using Flume. 

·         Automating and scheduling the Sqoop jobs in a timely manner using Unix Shell Scripts.

·         Configured and Installed Hive on the Hadoop cluster. 

·         Handled importing of data from various data sources, performed transformations using Hive, Map Reduce, and loaded data into HDFS. 

·         Analyzed the data by performing Hive queries (HiveQL) and running Pig Latin scripts to study customer behaviour. 

·         Wrote Java code to format XML documents; upload them to Solr server for indexing. 

·         Using Hive join queries to join multiple tables of a source system and load them to Elastic search tables.

·         Written various key queries in Elastic search for retrieval of data effectively. 

·         Extensively worked on Elastic search querying and indexing to retrieve the documents in high speeds. 

·         Written Elastic search template for the index patterns. 

·         Loaded two different datasets sources like Oracle, MySQL to HDFS and Hive respectively on daily basis.

·         Optimized Hive queries to extract the customer information from HDFS or HBase.

·         Developed Pig Latin scripts to aggregate the log files of the business clients.

·         Designed and developed Pig Latin Scripts to process data in a batch to perform trend analysis. 

·         Using Hive and Pig to develop complex Map reduce streaming jobs using Java language.

·         Loaded and transformed large sets of structured, semi structured data using Pig Scripts.

·         Involved in writing shell scripts in scheduling and automation of tasks.

·         Managed and reviewed Hadoop log files to identify issues when Job fails.

·         Used Teradata Data Mover to copy data and objects such as tables and statistics from one system to another.

·         Worked on Teradata Multi-Load, Teradata Fast-Load utility to load data from Oracle and SQL Server to Teradata

Environment: HDP2.0, Hadoop, HDFS, Map Reduce, Pig, Hive, Sqoop, Java, Flume, Oracle, MySQL, HBase, Elastic Search, Linux Shell Scripting and Big Data.

 

Client: Quicken Loan, Houston, TX                                                    Jan 2014 – Mar 2015

Role:  Hadoop Developer 

 

Responsibilities:

·         Installed and configured Hadoop Map Reduce, HDFS, developed multiple Map Reduce jobs in java for data cleaning and pre-processing. 

·         Continuously monitored and managed the Hadoop Cluster using Cloudera Manager.

·         Used Cloudera manager to pull metrics on various cluster features like JVM, Running Map and reduce tasks. 

·         Utilized Apache Hadoop environment by using Cloudera.

·         Developed a workflow using Oozie to automate the tasks of loading the data into HDFS from analyzing the data.

·         Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box and analyzed the weblog data using the Hiveql.

·         Used Oozie work flow for Scheduling and managing jobs on a Hadoop cluster.

·         Developed complex queries using Hive and Impala.

·         Involved in working with Impala for data retrieval process. 

·         Experienced in migrating HiveQL into Impala to minimize query response time.

·         Performed transformations using Map Reduce, Hive to load data into HDFS and worked on importing data from various sources.

·         Developed several new Map Reduce programs to analyze and transform the data to uncover insights into the customer usage patterns.

·         Developed simple and complex Map Reduce programs in Java for Data Analysis. 

·         Written various Hive and Pig scripts and responsible for building scalable distributed data solutions using Hadoop.

·         Developed Hive UDFs to validate against business rules before data move to hive tables.

·         Used Pig scripts to join different data sets to perform queries.

·         Experienced in writing Pig UDF’s to perform analytics with Pig Latin operations.

·          Effectively used Sqoop to transfer data from databases (MySQL, Oracle) to HDFS, Hive. 

·         Analyzed relational databases using Sqoop to export data and generate reports.

·          Worked extensively with Sqoop for importing metadata from Oracle. 

·         Used MFT to work on the Ingestion of Files into HDFS from remote systems.

·         Worked on Avro Data files using Avro Serialization system.

·         Experienced with different scripting language like Python and shell scripts.

·         Implemented Frameworks using Java and python to automate the ingestion flow.

·         Designed & Implemented database Cloning using Python and Built backend support for Applications using Shell scripts

 

Environment: HDFS, Map Reduce, Pig, Hive, Sqoop, Python, Maven, Avro, Impala, Oozie, Avro, Eclipse and Shell Scripting.

 

Client: Mind tree, India                                                                     Mar 2012 – Nov 2013

Role:  Java Developer 

 

Responsibilities:

·         Implemented business rules by using Java & J2EE technology to modify the components.

·         Involved in Agile methodology includes daily SCRUM meetings, iteration planning, etc.

·         Designed different use cases by using UML, Class and Activity diagrams.

·         Developed GUI application by using HTML, Java Script and JSP to view pages in desktop portal.

·         Implemented Core Java in the application by using (Multi-Threading, Collections, OOPS, exception handling, JDBC).

·         Developed Spring ORM/JDBC layers to integrate Hibernate for regular JDBC calls.

·         Implemented navigation layer using Spring MVC components like dispatcher Servlet, controllers and view resolver components.

·         Implemented business components as Spring Beans and configured using Dependency Injection and annotations.

·         Consumed Web Services (REST) from third party and implemented those using JAX-RS.

·         Used Hibernate in persistence layer and developed POJO's, Data Access Object (DAO) to handle all database operations.

·         Packaging and Deployment of builds through Maven scripts.

·         Deployed the Application on Web Sphere application Server and used Message Driven Beans with MQ Series.

·         Implemented logger for debugging using Log4j and used GIT as version management tool.

·         Performed Code review and JIRA is used to track the information.

·         Performed unit testing using JUNIT framework to check the code review.

·         Created and modified Stored Procedures, Functions, Triggers and Complex SQL Commands using Oracle 11g as a database.

Environment: Java SE/ J2EE , Core Java, JSP,JSTL, XML,HTML ,Query, Spring MVC/ DI /ORM /JDBC, Log4J, JUNIT, JavaScript, JSF, Ajax, Hibernate, Web Sphere, Eclipse, Unix, Ant, JUNIT, REST, GIT, Maven, MQ, JIRA, Oracle 11g.

 

Client: ADP, India                                                                                   Oct 2010 – Feb 2012

Role: Java Developer 

 

Responsibilities:

·         Coordinated in Object Oriented Analysis & Design of the project.

·         Developed HTML navigation menu that is derived from the database in the form of XML.

·         Performed client side validation for the application using Java-Script and J-Query.

·         Extensively involved in developing Web interface using JSP, JSP Standard Tag Libraries (JSTL) using Struts Framework. 

·         The Struts Framework is used in the application which is based on MVC design pattern.

·         Developed client-side AJAX application that uses XSLT, XPath, JavaScript, OOP, and retrieve them via JNDI interface.

·         The Communication with databases is used by using JDBC and worked on Stored Procedures.

·         Used web services as a REST Services (JAX-RS) for XML response type.

·         Implemented Log4J for Logging Errors, debugging and tracking. 

·         Performed unit testing using JUNIT framework to check the code review.

<span style="font-size:11pt;font-family:Sym