Candidates About

 

 

 

Professional Summary

Hadoop Stack

·         8+ years of IT experience in software development, big data management, data modeling, data integration, implementation and testing of enterprise class systems spanning big data frameworks, advanced analytics and Java/J2EE technologies.

·         3+ years of hands on experience in Hadoop components & Map Reduce programming for parsing and populating tables for Terabytes of data.

·         Extensive usage of Sqoop, Flume, Oozie for data ingestion into HDFS & Hive warehouse.

·         Experienced on major Hadoop ecosystem’s projects such as Pig, Hive and HBase.

·         Good working experience using Sqoop to import data into HDFS from RDBMS and vice-versa.

·         Hands on performance improvement techniques for data processing in Hive, Impala, Spark, Pig & map-reduce using methods including but not limited to dynamic partitioning, bucketing, file compression.

·         Experience data processing like collecting, aggregating, moving from various sources using Apache Flume and kafka.

·         Expertise of ingesting data to Solr from HBase

·         Extensively worked on debugging using Eclipse debugger.

·         Experienced in importing data from different sources using StreamSets.

·         Experience with Cloudera, Hortonworks & MapR Hadoop distributions.

·         Strong work ethic with desire to succeed and make significant contributions to the organization.

·         Strong problem-solving skills, good communication, interpersonal skills and a good team player.

·         Have the motivation to take independent responsibility as well as ability to contribute and be a productive team member.

·         A pleasing personality with the ability to build great rapport with clients and customers.

·         Illustrates excellent verbal and written communication, paired with great presentation and interpersonal skills.

·         Portrays strong leadership qualities, backed with a great track record as a team player.

·         Adept with the latest business/technological trends.

 

Spark & Transaction Processing

·            Hands on experience with Spark-SQL for various business use-cases.

·            Used Spark-SQL, Scala APIs for querying & transformation of data residing in Hive.

·            Used python for Spark SQL jobs to fast process the data.

·            Replaced existing MR jobs with Spark streaming & Spark data transformations for efficient data processing.

 

TECHINICAL SKILLS

Hadoop Ecosystems                :           HDFS, MapReduce, YARN, Hive, Pig, Sqoop, Oozie, Flume, Spark and

                                                                 Zookeeper, Solr, StreamSets.

Apache Spark                          :           Spark, Spark SQL, Spark Streaming, Scala.

ETL Tools                                            :            Informatica with Hadoop connector, Pentaho, Alteryx

Scripting Languages                 :           Java, C, Scala, SQL, Unix Shell Scripting, Python

Java Technologies                   :           JQuery, JSP, Servlets.

SQL Databases                        :           Oracle, SQL Server 2012, SQL Server 2008 R2, DB2, Teradata

No-SQL                                   :           MongoDB, HBase.

Development tools                   :           Maven, Eclipse, IntelliJ, PyCharm

 

 

Educational Summary

Masters in Computer Technology from Eastern Illinois University (2015)

Bachelors of Science in Computer Science from Mahatma Gandhi Kashi Vidyapith (2009)

 

Courses & Workshops Attended

Qlik Sense Visual Analytics App Development Workshop – Charlotte, NC 2016

 

 

 

 

 

Professional Experience

 

Bank of America, Charlotte, NC                                                                                  Mar 17 – Present    

Role: Hadoop Developer

 

Global Information Security is part of Cyber Security applications which maintains the information of different kinds of sources such as house address, social security related information, bank transaction details etc. I was involved for data analysis and Real time streaming the data which comes from different kinds of applications.

 

 

Responsibilities:

·         Created external hive tables to move the data from different source to cloudera.

·         Used to keep track of the data once it gets loaded and updated on Weekly and daily basis

·         Performed SQL Joins among Hive tables to make it into one table.

·         Ingested data from Hive to HBase and HBase to Solr using Spark.

·         Worked on Ingesting the data using StreamSets from different sources like JDBC to Hive by Sqoop jobs.

·         Data import from Hive to Solr using StreamSets. 

·         Near-Real time indexing into Solr for automated process after scheduling the job.

·         Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python.

·         Involved in developing ETL data pipelines for real-time streaming data using Kafka and Spark.

·         Worked on POC to pull over the third-party data and used Spark SQL to create schema RDD and loaded it into Hive Tables and structured data using Spark SQL.

·         Importing and exporting data into HDFS, Pig, Hive and HBase using Sqoop.

·         Managing and reviewing Hadoop log files.

·         Worked on loading and transformation of large sets of structured, semi structured and unstructured data into Hadoop system.

·         Responsible to manage data coming from different data sources.

·         Developed Flume ETL job for handling data from HTTP source and sink as HDFS.

·         Worked closely with Admin to setup with Kerberos Authentication.

·         Write test cases to test software throughout development cycles, inclusive of functional/unit-testing/continuous integration.

·         Manage operations, monitoring, and troubleshooting for all HADOOP development and production issues.

·         Develop/Debug performance testing/tuning in the existing application.

·         Detailed design specification document and implementing business rules.

·         Worked on loading and transformation of large sets of structured, semi structured and unstructured data into Hadoop system.

·         Analyzed large data sets by running Hive queries and Pig scripts.

·         Involved in creating Hive tables, and loading and analyzing data using hive queries.

·         Experienced in running Hadoop streaming jobs to process terabytes of xml format data.

·         Gained experience in managing and reviewing Hadoop log files.

·         Interacted closely with Web Developer for usage of application and to pull data from Solr as well as HBase and populate it in front end.

·         Configured Spark streaming to receive real time data from Kafka and store the stream data to HDFS using Scala.

·         Used Spark-SQL to Load JSON data and create Schema RDD and loaded it into Hive Tables and handled Structured data using SparkSQL.

·         Implemented Spark SQL queries using Python for fast processing the data.

·         Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map way.

·         Exported analyzed data to relational databases using Sqoop for visualization to generate reports for the BI team.

 

 

Environment: Hive, HDFS, HBase, Solr, StreamSets, Spark, Kafka,Sqoop, Scala, IntelliJ, Python, PyCharm.

 

 

Sparsity Systems, Charlotte, NC                                                                                  Jan 15 – Feb 17      

Role: Hadoop Developer

 

 

Utilized the Hadoop cluster for staging & analyzing data from various data sources including web service API’s, relational databases and text files from an Insurance database containing policy related information, new quotes information for new and existing policy holders.   

 

Responsibilities:

·         Developed data pipeline using Sqoop, Flume to store data into HDFS and further processing through spark.

·         Creating Hive tables with periodic backups, writing complex Hive/Impala queries to run on Impala.

·         Implemented partitioning, bucketing and worked on Hive, using file formats and compressions techniques with optimizations.

·         Wrote Python script to convert Autosys jobs, HDFS directory location paths from old standards to new standards.

·         Wrote Python scripts for getting yarn job list for performance metrics.

·         Created Hive Generic UDF's to process business logic that varies based on policy.

·         Experience in customizing map reduce framework at different levels like input formats, data types, custom serde and partitioners.

·         Pushed the data to Windows mount location for Tableau to import it for reporting.

·         Continuous monitoring and managing the Hadoop cluster using Cloudera Manager.

·         Implemented Spark to migrate map reduce jobs into Spark RDD transformations, streaming data using Spark streaming

·         Worked on joins to create Hive look up tables.

·         Developed Spark scripts using Scala, Spark SQL to access hive tables in spark for faster data processing.

·         Analyzed large data sets by running Hive queries scripts.

·         Involved in creating Hive tables, and loading and analyzing data using hive queries

·         Developed HIVE scripts passing dynamic parameters using hivevar.

·         Created partitioned tables in Hive for best performance and faster querying.

·      Configured build scripts for multi module projects with Maven.

·      Automated the process of scheduling workflow using Oozie and Autosys. 

·         Prepared Unit test cases and performed unit testing.

·         Created external table and partitioned tables in hive for querying purpose.

 

 

Environment: Hadoop, Cloudera, HDFS, Hive, Spark, Sqoop, Flume, Java, Scala, Shell-script, Impala, Eclipse, Tableau, MySql.

 

 

 

 

 

Allegiant Info Services, Hyderabad, India                                                                      June 09 - Dec 13

Role: Java Developer

Description: Project details is to create the application is to get the track of transactions and the medical record of the patient when searching with the patients first and last name and to get the overall bill of the patient including health related issues and once the patient again comes to the hospital, then it shows all the concerns of the previous health issues including the medical records.

 

Responsibilities:

·         Involved in requirement Analysis, Designing, Coding and Testing.

·         Developed application on Agile scrum basis.

·         Developed and implemented the MVC Architectural pattern using Struts Framework including JSP, Servlets, EJB and Action classes.

·         Object Oriented Analysis and Design using UML include development of class diagrams, Sequence diagrams and State diagrams and implemented these diagrams in Microsoft Visio.

·         Involved in writing client-side validations using JavaScript, CSS.

·         Designed and developed the UI using Struts view components HTML, CSS and JavaScript.

·         Developed JMS API using J2EE package.

·         Used Oracle as Database and used Toad for queries execution and involved in writing SQL scripts, PL/SQL code for procedures and functions.

·         Involved in designing test plans, test cases and overall Unit testing of the system.

·         Prepared documentation and participated in preparing user's manual for the application.

 

Environment: Java, JQuery, Junit, Servlets, Spring 2.0, Web Logic, Eclipse, JSP, Windows XP, HTML, CSS, JavaScript, and XML.