Candidates About

 

 

Ziauddin

Sr. Big Data/Hadoop Engineer

Summary:

·         Around 9+ years of professional IT experience which includes experience in Big Data ecosystem and Java/J2EE related technologies.

·         Hands on experience Technical Designing and Developing BIG DATA solutions using Hadoop Ecosystems.

·         Experience in working with Flume to load the log data from multiple sources directly into HDFS.

·         Worked and learned a great deal from Amazon Web Services (AWS) Cloud services like EC2, S3, EBS, RDS and VPC.

·         Set up of Cluster servers on AWS and management of cluster servers.

·         Experienced with migrating data to and from RDBMS and unstructured sources into HDFS using Sqoop.

·         Handling data in various file formats such as Sequential, AVRO, RC, Parquet and ORC.

·         Experienced working on EC2 (Elastic Compute Cloud) cluster instances, setup data buckets on S3 (Simple Storage Service), set EMR (Elastic MapReduce).

·         Experienced in extending Hive and Pig core functionality by writing custom UDFs using Java.

·         years of experiences in big data development with Hadoop ecosystem, Hadoop, Spark, Hive, HDFS, HBase, Cassandra, Sqoop, Flume, Kafka, Zookeeper, and Oozie

·         Worked on real-time, in-memory processing engines such as Spark, Impala and integration with BI Tools such as Tableau.

·         Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.

·         Experience using different file formats like Binary, XML, JSON and CSV files. Experience in developing solutions to analyze large data sets efficiently.

·         Extensive experience in developing MYSQL, DB2 and Oracle Database Triggers, Stored Procedures and Packages within quality standards using SQL and PL/SQL.

·         Intensive work experience in developing enterprise solutions using Java, J2EE, Agile, Servlets, JSP, JDBC, Struts, spring, Hibernate, JavaBeans, JSF, MVC, JMS.

·         Comprehensive knowledge in Debugging, Optimizing and Performance Tuning of DB2, Oracle and MYSQL databases.

·         Experience with Cloudera and Horton Works distributions.

·         Expertise in Core Java, J2EE, Multithreading, JDBC, Shell Scripting and proficient in using Java API's for application development.

·         Experienced in using IDEs and Tools like Eclipse, Net Beans, GitHub, Jenkins, Maven and IntelliJ.

·         Extensive Experienced in Unit Testing with JUnit, MRUnit, Pytest

·         Experience using middleware architecture using Sun Java technologies like J2EE, JSP, Servlets, and application servers like Web Sphere and Web logic.

·         Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.

·         Good Inter personnel skills and ability to work as part of a team. Exceptional ability to learn and master new technologies and to deliver outputs in short deadlines.

 

Technical Skills:

·         Big data/Hadoop: HDFS, Map Reduce, Hive, Pig, Sqoop, Oozie, Hue, Flume, Kafka and Spark

·         NoSQL Databases: HBase, MongoDB & Cassandra

·         Java/J2EE Technologies: Servlets, JSP, JDBC, JSTL, EJB, JAXB, JAXP, JMS, JAX-RPC, JAX- WS

·         Web Tools: HTML, Java Script, XML, ODBC, JDBC, Hibernate, JSP, Java, Struts, spring, Junit, JSON and Avro

·         Programming Languages: Java, Python, SQL, PL/SQL, AWS, HiveQL, Unix Shell Scripting, Scala

·         Methodologies: Software Development Lifecycle (SDLC), Waterfall Model and Agile, STLC (Software Testing Life cycle) & UML, Design Patterns (Core Java and J2EE)

·         Database: Oracle 12c/11g, MYSQL , SQL Server 2016/2014

·         Web/ Application Servers: Weblogic, Tomcat, JBoss

·         Web Technologies: HTML4/5, CSS3/2, XML, JavaScript, JQuery, AJAX, WSDL, SOAP

·         Tools and IDE: Eclipse, NetBeans, Maven, DB Visualizer, Visual Studio 2008, SQL Server Management Studio

 

John Hancock - Boston, MA                                                                         Aug 16  - till date

Sr. Big Data/Hadoop Engineer

Responsibilities:

·         Developed MapReduce programs in Python and Scala for processing the extracted customer data.

·         Created shell scripts for automating the process of extracting and loading targeted customers into Hive tables on daily basis to whom various email campaigns are sent.

·         Developed and deployed Hive UDF's written in Java for encrypting customer-id's, creating item-image-URL's etc.

·         Extracted Strong View and Kafka logs from servers using Flume and extracted information like open/click info of customers and loaded into Hive tables.

·         Written Shell scripts for automation of Hive query processing.

·         Scheduled Map-Reduce and Hive workflows using Oozie and Cron.

·         Used Apache NiFi to generate graphical representation of data transferring and flow.

·         Developed HTML templates for various trigger campaigns.

·         Worked on Oracle PL/SQL and Teradata some extract and loading data.

·         Analyzed transaction data and extracted category wise best-selling items info which is used by marketing team to come up with ideas for new campaigns.

·         Developed complex Hive queries using Joins and automated these jobs using Shell scripts.

·         Developed Spark application for both batch process and also for streaming process.

·         Created Hive external tables with ORC, Bucketing & Transactional properties.

·         Monitored and debugged Map-Reduce jobs using the Job-tracker administration page.

·         Developed Storm Topologies for real time email campaigns where Kafka is used as source for getting customer's website activity information and storing data into Redis server.

·         Involved in migrating existing Hive jobs to Spark SQL environment.

·         Used Spark Streaming API for consuming data from Kafka source and processed data with core Spark functions written in Scala and then stored resultant data in HBase table which is later used for generating reports

·         Developed data pipeline to ingest data from Kafka source into HDFS as sink using Flume which is used for analysis. 

·         Created a repository in GitHub (version control system) to store project and keep track of changes to files.

·         Used Eclipse Neon to develop spark application in PYDEV perspective.

·         Used IDLE (python GUI) for developing python code and incorporate in spark application.

·         Developed REST web services for providing metadata information required for the campaigns.

Environment:MapReduce,Hive,HBASE,Python,Java,Storm,Scala,SparkStreaming,SparkSQL,Redis,Oozie,Kafka,Flume,REST web services, Tableau, Teradata.

CVS Health, Phoenix, AZ                                                                               Jun 15 – Jul 16

Sr. Big Data/Hadoop Engineer

Responsibilities:

·         Developed multiple MapReduce programs for analyzing the insurance data of the customer and produce summary results from Hadoop to downstream systems.

·         Automated workflows using shell scripts which pulls developed code from GitHub into Hadoop.

·         Developed multiple MapReduce programs for analyzing the insurance data of the customer and produce summary results from Hadoop to downstream systems.

·         Develop/capture/document architectural best practices for building systems on AWS.

·         Involved in Installing, Configuring Hadoop Eco System and Cloudera Manager using CDH4 Distribution.

·         Created Hive queries to compare the raw data with EDW reference tables and performing aggregates

·         Developed Map-Reduce programs in Python.

·         Importing and exporting data into HDFS and Hive using Sqoop.

·         Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.

·         Built wrapper shell scripts to hold these Oozie workflow.

·         Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.

·         Integrating bulk data into Cassandra file system using MapReduce programs.

·         Installed and configured Hive and also written Hive UDFs.

·         Performed SQL Joins among Hive tables to get input for spark batch process.

·         Active member for developing streaming application using Apache Kafka.

·         Ingested all formats of structured and semi-structured data including relational databases, JSON using NIFI & Kafka into HDFS.

·         Used TOAD for Querying Hive tables for faster execution.

·         Involved in creating Hive tables, loading with data and writing hive queries using the HiveQL which will run internally in map reduce way.

·         Extracted the data from MySQL, AWS Redshift into HDFS using Sqoop.

·         Deployed Hadoop Cluster in Fully Distributed and Pseudo-distributed modes.

·         Involved in managing and monitoring Hadoop cluster using Cloudera Manager.

·         Involved in daily SCRUM meetings to discuss the development/progress and was active in making scrum meetings more productive    

·         Supported in setting up QA environment and updating configurations for implementing scripts with Pig, Hive and Sqoop.

·         Unit tested a sample of raw data and improved performance and turned over to production.

·         Implemented Angular Js Controller functions, service using Controller methods to set up initial state of the object.

Environment: Hadoop, Map Reduce, ETL, Hive, HDFS, PIG, Scala, Agile, AngularJS, Python, Sqoop, Oozie, AWS, Cloudera, Flume, HBase, Zookeeper, CDH3, Oracle, NoSQL and Unix/Linux.

AIG - Houston, TX                                                                                          Apr 14 – May 15

Sr. Java /Hadoop Developer

Responsibilities:

·         Worked on capacity planning and management of HDFS Clusters to meet the project needs.

·         Worked on custom Pig Loaders and storage classes to work with variety of data formats such as JSON and XML file formats.

·         Installed Oozie workflow engine to run multiple Hive and Pig jobs, which run independently with time and data availability.

·         Developed data pipeline using Flume, Sqoop, Pig and Java MapReduce to ingest customer behavioral data and financial histories into HDFS for analysis.

·         Handled importing of data from various data sources, performed transformations using Hive, MapReduce and loaded data into HDFS.

·         Developed custom tags , JSTL to support custom user interfaces

·         Involved in designing the user interfaces using JSPs and Servlets.

·         Developed presentation layer using HTML, CSS and Java script.

·         Used EXT-JS framework for building interactive web applications using techniques such as Ajax, DHTML and DOM scripting.

·         Used XML wed services using SOAP to transfer the amount to transfer application that is remote and global to different financial institutions.

·         Involved in development of web services for business operations using various Web Services API and tools like SOAP, WSDL, JAX-WS, JDOM, XML and XSL.

·         Developed automated processes for flattening the upstream data from Cassandra which in JSON format. Used Hive UDFs to flatten the JSON Data.

·         Involved in developing Pig UDFs for the needed functionality that is not out of the box available from Apache Pig.

·         Involved in processing ingested raw data using MapReduce, Apache Pig and Hive.

·         Involved in writing the hive queries for analysis on the structured data in the output folder of HDFS

·         Developed application using spring framework that leverages MVC (model view layer architecture).

·         Developed business domain layer using session and entity beans EJB.

·         Used Java Messaging Services (JMS) for reliable and asynchronous exchange of important information such as payment status report.

·         Worked with a variety of issues involving multi-threading, server connectivity and user interface.

·         Made extensive use of java Naming and Directory interface (JNDI) for looking up of enterprise beans.

·         Developed SQL, PL/SQL, stored procedures - database application scripts.

·         Involved in Sprint meetings and followed agile software development methodologies.

·         Deployed the application on Web logic Application Server.

·         Developed JUnit test cases for all the developed modules.

Environment: JDK1.7, Java, Hadoop, MapReduce, HDFS, Hive, Sqoop, HBase, Pig, Oozie, PL/SQL, JSP, PL/SQL, Spring , EJB , JMS,  XML, UML, SOAP, Rationale Rose, eclipse, Web Logic, Hibernate, MS SQL Server,

Comcast - Philadelphia, PA                                                                          Jan 11 – Mar 14

Sr. Java/J2EE Developer

Responsibilities:

·         Implemented various J2EE Design Patterns such as Model-View-Controller (MVC), Data Access Object, Business Delegate and Transfer Object

·         Involved in requirement gathering, functional and technical specifications. 

·         Fixing the existing bugs in various releases.

·         Wrote requirements and detailed design documents, designed architecture for data collection.

·         Developed OMSA UI using MVC architecture, Core Java, Java Collections, JSP, JDBC, Servlets and XML within a Windows and UNIX environment.

·         Used Java Collection Classes like Array List, Vectors, Hash Map and Hash Table.

·         Created Hibernate OR mapping of the tables and integrated with spring (Transaction Management).

·         Used Maven for building the application, and completed testing by deploying on application server

·         Experienced building Restful (AJAX/JSON) applications. 

·         Developed algorithms and coded programs in Java.

·         Designed various tables required for the project in Oracle 10g database and involved in coding the SQL Queries, Stored Procedures and Triggers in the application.

·         Pushed the code to Jenkins and integrated the code with Maven. 

·         Developed Unit Test Cases. Used JUNIT for unit testing of the application.

·         Involved in design and implementation using Core Java, Agile, Struts, and JMS.

·         Developed complex SSAS cubes with multiple fact measures groups, and multiple dimension hierarchies based on the OLAP reporting needs.

·         Performed all types of testing includes Unit testing, Integration and testing environments.

·         Used XML for Data presentation, Report generation and customer feedback documents.

·         Developed JUnit test cases for regression testing and integrated with Maven build.

·         Worked on a modifying an existing JMS messaging framework for increased loads and performance optimizations.

·         Used Combination of client and server side validation using Struts validation framework.

·         Use of Joins, Triggers, Stored Procedures and Functions in order to interact with backend database using SQL.

·         Coordinated with other Development teams, System managers and web master and developed good working environment

 

Environment: JAVA, Agile, JSP, JSON, HTML, , Eclipse IDE Ajax, Design Patterns, struts, spring, Hibernate, Oracle, SQL/ PL SQL, JMS.

 

Valuelabs – Hyderabad, IN                                                                           Jul 08 – Dec 11

Java/J2EE Developer

Responsibilities

·         Involved in development of business domain concepts into Use Cases, Sequence Diagrams, Class Diagrams, Component Diagrams and Implementation Diagrams

·         Designed and developed Web Services using Java/J2EE in WebLogic environment.

·         Developed web pages using Java Servlet, JSP, CSS, Java Script, DHTML, and HTML. Added extensive Struts validation. Wrote Ant scripts to build and deploy the application.

·         Involve in the Analysis, Design, and Development and Unit testing of business requirements.

·         Responsible for developing platform related logic and resource classes, controller classes to access the domain and service.

·         Developed UI Using HTML, CSS and developed Business Logic and Interfacing components using Business Objects, XML, and JDBC.

·         Implemented business logic and generated WSDL for those web services using SOAP. 

·         Developed the application using Spring Web MVC framework.

·         Worked with Spring Configuration files to add new content to the website.

·         Worked on the Spring DAO module and ORM using Hibernate. Used Hibernate Template and Hibernate DAO Support for Spring-Hibernate Communication.

·         Configured Association Mappings such as one-one and one-many in Hibernate

·         Developed Servlets and JSP based on MVC pattern using Struts Action framework.

·         Involved in writing Hibernate queries and Hibernate specific configuration and mapping files.

·         Frequently wrote SQL on PL/SQL Developer to update and retrieve data from Oracle database

·         Developed unit test cases with JUnit

·         Used Log4J logging framework to write Log messages with various levels.

·         Involved in fixing bugs and minor enhancements for the front-end modules.

·         Coded various classes for Business Logic Implementation. Developed and deployed UI layer logics of sites using JSP.

·         Worked with Struts MVC objects like Action Servlets, Controllers, and Validators, Web Application Context, Handler Mapping, Message Resource Bundles and JNDI for look-up for J2EE components.

 

Environment: J2EE, JDBC, Java, Servlets, JSP, Struts, Hibernate, Web services, SOAP, WSDL, Design Patterns, MVC, HTML, JavaScript, WebLogic, XML, JUnit, Oracle 9i, My Eclipse.