Candidates About

 

Seshagiri Harsha

Summary:

·         9+ years of professional experience in IT, including 5 + years of work experience in Big Data, Hadoop Development and Ecosystem Analytics in Financial, Communication, retail and Pharmacy.

·         Good knowledge of Hadoop Architecture and various components such as HDFS, Resource Manager, Node Manager, Application Master, Container, Data Node, Name Node and Map-Reduce concepts.

·         Expertize with the tools in Hadoop Ecosystem including Hive, HDFS, MapReduce, Sqoop, Spark, Kafka, Yarn, Oozie, Flume and Zookeeper.

·         Hands on experience with Spark streaming to receive real time data using Kafka

·         Good knowledge of No-SQL database HBASE, Cassandra and Elasticsearch.

·         Good knowledge on scripting languages like PYTHON, SCALA.

·         Efficient in writing MapReduce Programs and using Apache Hadoop Map Reduce API for analyzing the structured and unstructured data.

·         Experience in using Apache Sqoop to import and export data to and from HDFS and external RDBMS databases.

·         Hands on experience in setting up workflow using Apache Oozie workflow engine for managing and scheduling Hadoop jobs.

·         Expertise in writing queries using HIVE Dynamic Partitioning and Bucketing.

·         Experienced in using Aggregate functions, table generated functions, implementing UDF's to handle complex objects.

·         Experienced in handling different optimization join operations like Map join, Sorted Bucketed Map join, etc.

·         Experienced in Tuning Hive queries using Hive configurable parameters.    

·         Experienced in proving User based recommendation or predictive analytics using Spark Mlib, Mahout library.

·         Experienced in writing queries using SparkQL.

·         Experience in written Programs using Scala.

·         Hands on experience on tools such as Eclipse, Intelij, Toad, Linux vi-editor and project management tools like Jira and Rally.

·         Experience in writing Shell scripts using ksh, bash, and python, for process automation of databases, applications, backup and scheduling

·         Ability to work non-peak hours when needed, assist in capacity planning, proactively monitor performance, Collaborate with engineering teams to optimize data collection.

·         Maintain change control and testing processes for all modifications and deployments.

·         Conduct research and make recommendations on big data products, services, and standards

·         Backup and recovery of HDFS file system

·         Experience setting up Kerberos and administering a kerberized cluster.

• Involved in grooming sessions with Project Manager/Scrum Master/Technology Director.

·         Knowledge of data warehousing and ETL tools like Informatica, Ab into, Datastage.

• Experience in Visualization tool Tableau

·         Detailed understanding of Software Development Life Cycle (SDLC) and sound knowledge of project implementation methodologies including Waterfall and Agile.

·         Hands-on programming experience in various technologies like JAVA, JSP, Servlets, SQL, JDBC, HTML, XML, UNIX.

Education:

 

·         Bachelor of Engineering in Computer Science from JNTU, India , 2007

·         M.S from St. Cloud State University, Minnesota 2014

 

Technical Skills:

 

Hadoop Technologies and Distributions	Cloudera Hadoop Distribution CDH4, CDH5, MapR, Horton works Data Platform (HDP) and Azure
Hadoop Ecosystem	HDFS, Hive, Sqoop, Oozie, Flume, Spark streaming, Zookeeper, Map-Reduce, Storm, Kafka, Redis
NoSql Databases	HBase, Cassandra, Mongo DB, PIG, Elastic search, AWS
Programming	C, C++, Python, Scala, Java, PL/SQL, Algorithms, R
RDBMS	ORACLE, MySQL, SQL Server, Redshift, Netezza, Teradata
IDE	Eclipse4.x, Intellij
Version and Build and Release	Git, Maven, SBT, Jenkins
Operating Systems	Linux (RedHat, CentOS), Windows XP/7/8, 2012R2
Web Servers	Apache Tomcat, Node
Cluster Management Tools	Cloudera Manager, Horton Works Ambari
BI	Tableau, Microstrategy

 

Professional Experience:

 

Visa Inc., Palo Alto, CA                                                                                             Jan 2017 – till date

Big Data Tech Lead / Developer                                           

 

VISA is an American multinational financial services corporation facilitates electronic funds transfers throughout the world, most commonly through Visa-branded credit cards and debit cards. Visa does not issue cards, extend credit or set rates and fees for consumers; rather, Visa provides financial institutions with Visa-branded payment products that they then use to offer credit, debit, prepaid and cash-access programs to their customers.

Responsibilities:

 

• Worked on hybrid role as big data developer and Tableau report developer.
• Wrote technical design document, deployment document, supporting documents.
• Initiated agonistic data ingestion concept, which does automated scripts based on the schema changes.
• Provided support during deployments and helped admin team in end point changes.
• Daily status check for Oozie workflow and implement necessary changes in case of tweak’s and monitor Cloudera manager and check data node status to ensure nodes are up and running.
• Worked on historical load for the various sources.
• Involved in grooming sessions with Project Manager/Scrum Master/Technology Director.
• Extensively helped stakeholders and guided them for revenue benefits and business model implementations with partners.
• Experienced in handling different data formats like Json in Hive using Hive SerDe's.
• Experienced with different compression techniques like LZO, GZip and Snappy.
• Extensively worked on Hive tables, partitions and buckets for analyzing large volumes of data
• Worked on Map Reduce jobs for converting XML data into JSON data format and store on HBase
• Used spark as ELT tool to do transformations, joins and aggregations before storing the data into HDFS
• Developed shell scripts for adding process dates to the source files
• Implemented the workflows using Oozie.
• Converted Impala scripts into self-service where analytics team can pull the data.
• Lead teammates for converting business onetime reports to self-services solutions and automated reports.
• Worked with Project Managers to ingest data into Hadoop ecosystems from all possible sources within the organization and laid platform for analytics/R team.
• Worked on more financial reports, which stood me up as go to guy for business.
• Worked on data lake concepts, converted all ETL jobs into hive scripts.
• Reverse engineered the reports and identified the Data Elements (in the source systems), Dimensions, Facts and Measures required for new enhancements of reports
• Creating Dashboards/Reports/Domain/Adhoc View and built Data visualizations using Tableau and provide analysis on the data.
• Created audit report which shows all users usage on reports.
• Suggested improvement processes for all process automation scripts and tasks.
• Involved in the technology and architectural meetings across all teams and provided inputs to other teams.

 

Environment: Cloudera,MapReduce,HDFS,Pig,Hive,Sqoop,Oozie,SQLserver,tableau,Sourcetree,stash,Intellij,JIRA.

 

 

Polariswireless, Mountain view, CA                                                                        Mar 2016 – Dec 2016

Hadoop Lead Developer

 

In 1999, Polaris Wireless was founded by Silicon Valley visionaries who invented, developed, and patented the high accuracy, software-based Wireless Location Signatures (Polaris WLS™) location technology. After successfully performing multiple field trials of Polaris WLS in the US and Europe, Polaris Wireless was awarded its first public safety contract with SunCom (now part of T-Mobile USA) in 2003. After expanding its deployments across 24 wireless operator networks in the US, in 2008 the company entered the field it would soon come to dominate, winning its first location surveillance deployment in the Asia-Pacific region.

Responsibilities: 

·         Giving estimates to the stake holders project managers and team members about the tasks

·         Maintaining offshore and onshore team and updating progress to business and management

·         Involved in developing algorithm for data moniterization using R

·         Responsible for architecting Hadoop clusters with CDH4 on CentOS, managing with Cloudera Manager.

·         Design & develop solution using Hadoop eco system tools

·         Responsible for design and developing scalable solution on distributed data using Hadoop eco system tools like Hive, Pig, Sqoop, Flume, storm, spark, Kafka and Oozie. 

·         Provide solution on architect in Hadoop data lake project.

·         Developed Simple to complex Map/reduce Jobs using java, Scala, python, Hive and Pig 

·         Experienced in creating several UDFs, UDAF, and UDTF for Hive using Java

·         Backup and recovery of HDFS file system

·         Used Maven extensively for building jar files of MapReduce programs and deployed to Cluster for continuous integration and continuous deployment

·         Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms 

·         Handled importing of data from various data sources, performed transformations using Hive, Map Reduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop 

·          Analyzed the data by performing Hive queries and running spark-Scala scripts to study customer behavior

·         Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and other sources. 

·         Use Flume to collect the log data from different resources and transfer the data type to hive tables using different SerDes to store in JSON, Avro, XML and sequence file formats.

·         Wrote the shell scripts to monitor the Hadoop daemon services and respond accordingly to any warning or failure conditions.

·         Continuous monitoring and managing the Hadoop cluster using Cloudera Manager 

·         Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required 

·         Developed Map-Reduce programs to process data that is stored in HDFS using Java. 

·          Loaded business accounts/customer details into HDFS from RDBMS Database using Sqoop. 

·         Involved in schema design for the application using Hive and HBase. 

·          Installed and configured Hadoop Map Reduce, HDFS (non-production environment).

·         Coordinate with other teams for each quarterly deployment and deploy new functionality in prod environment.

·         Techno-functional responsibilities include interfacing with users, identifying functional and technical gaps, estimates, designing custom solutions, development, leading developers, producing documentation, and production support.

·         Designed, documented operational problems by following standards and procedures using a software reporting tool JIRA. 

·         Production Rollout Support which includes monitoring the solution post go-live and resolving any issues that are discovered by the client and client services teams.

·         Worked on analyzing Hadoop stack and different big data analytic tools including Spark and Hive, HBase and Sqoop. 

 

Environment: Hadoop, HDFS, Hive, Pig, Sqoop, HBase, Hue, Linux, Map Reduce, Hadoop distribution of Cloudera and Flume.

 

 

Cisco Systems, San Jose, CA                                                                                    Jan 2015 – Feb 2016

Sr. Hadoop Developer                                                                                 

Designed and developed big data solutions involving Terabytes of data. The big data solution consists of collecting large amounts of log data from distributed sources, transformations and standardizations analysis, statistics, aggregations and reporting etc. Built an on demand elastic Hadoop cluster infrastructure to cater the needs of various Big Data projects, automated various Big Data workflows to process and extracts analytics out of the data using MapReduce,  and Hive.

Responsibilities:

·         Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables

·         Reverse engineer the old scripts and rewrite them in hive

·         Installed & maintained Cloudera Hadoop distribution

·         Experience setting up Kerberos and administering a kerberized cluster

·         Design & develop solution using Hadoop eco system tools like Flume, Hive, Kafka, Pig, Sqoop and Oozie on the Hadoop cluster.

·         Involved in loading the data from Linux file system to Datalake, Data validation and transformation in Data Lake environment.

·         Implemented Map Reduce programs on log data to transform into structured way to find user information.

·         Performed performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.

·         Exported the analyzed data to the relational databases using sqoop for virtualization and to generate reports for the BI team

·         Monitored workload, job performance and capacity planning using Cloud era Manager.

·         Installed Oozie workflow engine to run multiple Map Reduce, Hive and Pig jobs.

·         Responsible for creating Hive tables, loading the structured data resulted from MapReduce jobs into the tables and writing hive queries to further analyze the logs to identify issues and behavioral patterns.

·         Imported data frequently from MySQL to HDFS using Sqoop.

·         Supported operations team in Hadoop cluster maintenance activities including commissioning and decommissioning nodes and upgrades.

·         Used ETL tool, Talend to do transformations, event joins, filter and some pre-aggregations

·         Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.

·         Used Tableau for visualizing and to generate reports.

Environment: Hadoop, Cloudera, MapReduce, Hive, Sqoop, Flume, Talend, Python, MS-SQL Server, Tableau.

 

Charter Communications, St. Louis, MO                                                                        Jan 2014 - Dec 2015

BI Tableau / Hadoop ETL Developer

 

Charter Communications is an American telecommunications company, which offers its services to consumers and businesses under the branding of Spectrum. Providing services to over 25 million customers in 41 states, it is the second-largest cable operator in the United States by subscribers, just behind Comcast, and third largest pay TV operator behind Comcast and AT&T U-verse/DirecTV. It is the fifth largest telephone provider based upon residential subscriber line count.

 

• Identify new metrics and enhance the existing KPI's and New KPI score cards.
• Statistical Analysis of the data provided by the company to compare and analyze various measures including merchant categories, location and fraud detection.
• Analyzing the data using Statistical features in Tableau to develop trend analysis
• Data collection and leveraging on data manipulation teams for required data.
• Developed complex SQL scripts for Oracle database for creating BI layer on DW for tableau reporting.
• Building, publishing customized interactive reports and dashboards, report scheduling using Tableau server.
• Created action filters, parameters and calculated sets for preparing dashboards and worksheets using Tableau.
• Effectively used data blending, filters, actions, Hierarchies feature in tableau.
• Administered user, user groups, and scheduled instances for reports in Tableau.
• Executed and tested required queries and reports before publishing.
• Hands-on development assisting users in creating and modifying worksheets and data visualization dashboards.
•  

Environment: Tableau Desktop 7.0.2, Excel 2013, Outlook 2013, VMS Systems, Microsoft Office Suite (Word, Excel, Power Point, Visio).

 

Steria, Chennai, India                                                                                                            May 2007 - Dec 2012

Java / ETL Developer

 

The project data capturing application supports the creation and maintenance of order details for the Keystone and MPLS services provided by Purdue Pharma. These order details are forwarded to downstream application for provisioning of the requested services. This front-end application has interface with various Java applications and back end with Informatica which is used to send and receive information to third party vendors.

 

Responsibilities:

·         Involved in Full Life Cycle Development in Distributed Environment Using Java and J2EE framework.

·         Responsible for developing and modifying the existing service layer based on the business requirements.

·         Involved in designing & developing web-services using SOAP and WSDL.

·         Involved in database design.

·         Created tables, stored procedures in SQL for data manipulation and retrieval, Database Modification using SQL, PL/SQL, Stored procedures, triggers, Views in Oracle 9i.

·         Created User Interface using JSF.

·         Involved in integration testing the Business Logic layer and Data Access layer.

·         Integrated JSF with JSP and used JSF Custom Tag Libraries to display the value of variables defined in configuration files.

·         Written Stored Procedures functions and views to retrieve the data.

·         Used Maven build to wrap around Ant build scripts.

·         CVS tool is used for version control of code and project documents.

·         Responsible to mentor/work with team members to make sure the standards and guide lines are followed and delivery of tasks in time.

·         Implemented History Load, Incremental Load and ETL logic using Informatica 8.6/9.1 as per the ETL design document and Technical design document.

·         Familiar with ETL Standards and Process and developed ETL logic as per standards from Source-Flat File, Flat-File-Stage, Stage-Work, Work-Work Interim tables and Work Interim tables- Target Tables.

·         Fixing the issues from Source System to downstream data mart during the development process till the code goes live into Production.

·         Deploying the Code to QA as per the project plan which consists of DDL’s for Source-Target tables, ETL logic, JCT Entries, Collects Stats for 3NF Tables and migrating the code to QA using Kintana.

·         Prepared ETL Scripts for Data acquisition and Transformation. Developed the various mappings using transformation like source qualifier, joiner, filter, router, Expression and lookup transformations etc.

·         Doing Analysis for existing ETL Jobs and understanding the flow.

 

Environment: JQuery, JSP, Servlets, JDBC, HTML, JUnit, JavaScript, XML, SQL, Maven, Web Services, UML, WebLogic Workshop and CVS, Informatics