Candidates About

 

Ajeem Mohiuddin

Summary:

ü  Over 8 years of experience including 4 years of experience with Hadoop Ecosystem in installation and administrated of all UNIX/LINUX servers and configuration of different Hadoop eco-system components in the existing cluster project. 

ü  Extensive experience in installing, configuring and administrating Hadoop cluster for major Hadoop distributions like CDH5 and HDP.

ü  Hands on experience on Hortonworks and Cloudera Hadoop environments.   

ü  Experienced in setting up automated monitoring and escalation infrastructure for Hadoop Cluster using Ganglia and Nagios. 

ü  Experience with complete Software Design Lifecycle including design, development, testing and implementation of moderate to advanced complex systems. 

ü  Hands on experience in installation, configuration, supporting and managing Hadoop Clusters using Apache, Hortonworks, Cloudera and MapReduce. 

ü  Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa. 

ü  Hands on experience on configuring a Hadoop cluster in a professional environment and on Amazon Web Services (AWS) using an EC2 instance. 

ü  Excellent knowledge of in NOSQL databases like HBase, Cassandra. Experience in monitoring and troubleshooting issues with Linux memory, CPU, OS, storage and network.

ü  Experience in creating custom Lucene/Solr Query components.

ü  Expertise with the tools in Hadoop Ecosystem including Pig, Hive, HDFS, Map Reduce, Sqoop, Solr, Spark, Impala, Kafka, Flume, Yarn, Oozie, and Zookeeper. 

ü  Used NoSQL database with Cassandra, HBase, Hive, MongoDB and Monod.

ü  Experience in developing and scheduling ETL workflows in Hadoop using Oozie. Also have substantial experience writing MapReduce jobs in Java, Pig, Flume, Zookeeper and Hive and Storm. 

ü  Strong experience in System Administration, Installation, Upgrading, Patches, Migration, Configuration, Troubleshooting, Security, Backup, Disaster Recovery, Performance Monitoring and Fine-tuning on Linux (RHEL) systems. 

ü  Extensively involved in Test Plan re-design, Test Case re-Creation, Test Automation and Test Execution of web and client server applications as per change requests.

ü  Experience in working on internet related technologies such as HTML, JavaScript, VBScript and XML.

ü  Having Strong Experience in LINUX/UNIX Administration, expertise in Red Hat Enterprise Linux 4, 5 and 6, familiar with Solaris 9 &10 and IBM AIX 6. 

ü  Created POC to store Server Log data into Cassandra to identify System Alert Metrics.

 

Technical Skills

ü  Big Data Ecosystems: Hadoop, MapReduce, HDFS, HBase, Hive, Pig, Sqoop, Spark, Storm, Kafka, Oozie, MongoDB, Cassandra.

ü  Languages: C, Core Java, Unix, SQL, Python, R, C#, Haskell, Scala.

ü  Methodologies: Agile, UML, Design Patterns (Core Java and J2EE).

ü  Monitoring and Reporting: Ganglia, Nagios, Custom Shell scripts.

ü  NoSQL Technologies: Cassandra, MongoDB, Neo4j, Hbase.

ü  Frameworks: MVC, Struts, Hibernate, And Spring.

ü  Database: Oracle 11g, MySQL, MS-SQL Server, Teradata. PostgreSQL, IBM DB2

ü  Operating Systems: Windows XP/Vista/7, UNIX.

ü  Software Package: MS Office 2010.

ü  Tools & Utilities: Eclipse, Net Beans, My Eclipse, SVN, Git, Maven, SOAP UI, JMX explorer, XML Spy, QC, QTP, Jira.

ü  Web Servers: WebLogic, WebSphere, Apache Tomcat.

ü  Web Technologies: HTML, XML, JavaScript, jQuery, AJAX, SOAP, and WSDL.

Work Experience:

Sr. Hadoop Admin                                                                 Oct 2015 to present

HPE, Pleasanton CA                                                              

Responsibilities:

ü  Install and Manage HDP Hortonworks DL and DW components.

ü  Worked on Hadoop Hortonworks (HDP 2.2.0) distribution which managed services viz. HDFS, MapReduce2, Tez, Hive, Pig, Hbase, Cloudera, Sqoop, Flume, Spark, Ambari Metrics, Zookeeper, Falcon and Oozie etc.) for 4 cluster ranges from LAB, DEV, QA to PROD contains nearly 350+ nodes with 7PB data.

ü  Monitor Hadoop cluster connectivity and security on Ambari monitoring system.

ü  Led the installation, configuration and deployment of product soft wares on new edge nodes that connect and contact Hadoop cluster for data acquisition.

ü  Rendered L3/L4 support services for BI users, Developers and Tableau team through Jira ticketing system.

ü  One of the key engineers in Aetna's HDP web engineering team, Integrated Systems engineering ISE.

ü  Diligently teaming with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability. Worked on installing Kafka on Virtual Machine.

ü  Configured Kafka for efficiently collecting, aggregating and moving large amounts of click stream data from many different sources to MaprFS.

ü  Managing Amazon Web Services (AWS) infrastructure with automation and configuration management tools such as Chef, Ansible, Puppet, or custom-built designing cloud-hosted solutions, specific AWS product suite experience.

ü  Worked closely with System Administrators, BI analysts, developers, and key business leaders to establish SLAs and acceptable performance metrics for the Hadoop as a service offering.

ü  Worked with NoSQL databases like Hbase and Mongo DB for POC purpose.

ü  Configured, installed, monitored MapR Hadoop on 10 AWS EC2 instances and configured MapR on Amazon EMR making AWS S3 as default file system for the cluster.

ü  Extensively worked with Cloudera Distribution Hadoop, CDH 5.x, CDH4.x 

ü  Performance Tuning and ETL, Agile Software Deployment, Team Building & Leadership, Engineering Management.

ü  Hortonworks Ambari, Apache Hadoop on Redhat, and Centos as data storage, retrieval, and processing systems.

ü  Setting up Kerberos principals in KDC server and testing HDFS, Hive, Pig and MapReduce access for the new users and creating key tabs for service ID's using keytab scripts.

ü  Performed a Major upgrade in production environment from HDP 2.3 to HDP 2.6. As an admin followed standard Back up policies to make sure the high availability of cluster.

ü  Monitored multiple Hadoop clusters environments using Ganglia and Nagios. Monitored workload, job performance and capacity planning using Ambari.

ü  Installed OLAP software Atscale on its designated edge node server.

ü  Implemented dual data center set up for all Cassandra cluster. Performed much complex system analysis in order to improve ETL performance, identified high critical batch jobs to prioritize.

ü  Conducted cluster sizing, tuning, and performance benchmarking on a multi-tenant OpenStack platform to achieve desired performance metrics.

ü  Good knowledge on providing solution to the users who encountered java exception and error problems while running the data models in SAS script and Rscript. Good understanding on forest data models.

ü  Worked on data ingestion on systems to pull data scooping from traditional RDBMS platforms such as Oracle, MySQL and Teradata to Hadoop cluster using automated ingestion scripts and also store data in NoSQL databases such as HBase, Cassandra.

ü  Provided security and authentication with Kerberos which works on issuing Kerberos tickets to users.

ü  Good troubleshooting skills on over all Hadoop stack components, ETL services and Hue, Rstudio which provides GUI for developer’s/business users for day-to-day activities.

ü  Create queues and allocated the clusters resources to provide the priority for jobs in hive.

ü  Implementing the SFTP for the projects to transfer data SCP from External servers to servers. Experienced in managing and reviewing log files. Involved in scheduling Oozie workflow engine to run multiple Hive, Sqoop and pig jobs.

Environment: CDH 5.4.3 and 4.x, Cloudera Manager CM 5.1.1, HDFS, MapReduce, Yarn, Hive, Pig, Sqoop, Oozie, Apache, Impala, Kafka, Aws Flume, Zookeeper, Chef, Redhat/Centos 6.5, Control-M.

Hadoop Administrator                                                June 2014 to Aug 2015

Kellogg- Oak Brook, IL                                                

Responsibilities:

ü  Responsible for installing, configuring, supporting and managing of Hadoop Clusters.

ü  Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes.

ü  Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing log files.

ü  Monitored workload, job performance and capacity planning using Cloudera Manager.

ü  Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop.

ü  Used Hive and created Hive tables, loaded data from Local file system to HDFS.

ü  Created user accounts and given users the access to the Hadoop cluster.

ü  Performed HDFS cluster support and maintenance tasks like adding and removing nodes without any effect to running nodes and data.

ü  Migrated the existing data to Hadoop from RDBMS (SQL Server and Oracle) using Sqoop for processing the data.

ü  DBMS developments include building data migration scripts using Oracle SQL LOADER.  

ü  Responsible for HBase REST server administration, backup and recovery.

ü  Monitoring and controlling local file system disk space usage, log files, cleaning log files with automated scripts.

ü  As a Hadoop admin, monitoring cluster health status on daily basis, tuning system performance related configuration parameters, backing up configuration xml files.

ü  Monitored all MapReduce Read Jobs running on the cluster using Cloudera Manager and ensured that they were able to read the data to HDFS without any issues.

ü  Involved in upgrading Hadoop Cluster from HDP 1.3 to HDP 2.0.

ü  Experience in configuring Java components using Spark.

ü  Involved in moving all log files generated from various sources to HDFS for further processing.

ü  Extracted the data from Teradata into HDFS using the Sqoop.

ü  Involved in collecting metrics for Hadoop clusters using Ganglia and Ambary.

ü  Prepared Oozie workflow engine to run multiple Hive and Pig jobs which run independently with time and data availability.

ü  Supported Data Analysts in running MapReduce Programs.

ü  Developed Hive queries to process the data and generate the data cubes for visualizing.

ü  Responsible for deploying patches and remediating vulnerabilities.

ü  Experience in setting up Test, QA, and Prod environment.

ü  Involved in loading data from UNIX file system to HDFS.

ü  Created root cause analysis (RCA) efforts for the high severity incidents.

ü  Involved in Analyzing system failures, identifying root causes, and recommended course of actions.

ü  Worked hands on with ETL process. Handled importing data from various data sources, performed transformations.

ü  Coordinating with On-call Support if human intervention is required for problem solving

ü  Documenting the procedures performed for the project development.

Environment: Unix Linux Java, HDFS Map Reduce, Pig Hive HBase Flume Sqoop, NOSQL Oracle9i/10g/11g RAC with Solaris/red hat, Oracle Enterprise Manager (OEM), RMAN, Shell Scripting, Golden Gate, RedHat/Suse Linux, EM Cloud Control.

Hadoop Admin                                                             Sep 2013 to May 2014

Sears Outlet - Chicago, IL                                                    

Responsibilities:

ü  Installed/Configured/Maintained Apache/Cloudera Hadoop clusters for application development and Hadoop tools like MapReduce, Hive, Pig, HBase, Zookeeper and Sqoop.

ü  Responsible for building scalable distributed data solutions using Hadoop.

ü  Worked on building block of the C240 M3 Rack Servers, 2-RU server power by two Intel Xeon E5-2600 series processors, 256 GB of RAM, and 24 TB of local storage.

ü  Hands on experience in comprises of four racks, each containing 16 server nodes supporting 384 TB of raw storage per rack. This configuration can scale to 160 servers in a single management domain supporting 3.8 peta bytes of raw storage capacity.

ü  Developed and Modified Oracle Packages, Procedures, functions, Triggers as per the business requirements.

ü  Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.

ü  Managing and scheduling Jobs on a Hadoop cluster.

ü  Deployed Hadoop Cluster in Standalone Pseudo-distributed Fully Distributed modes

ü  Implemented NameNode backup using NFS. This was done for High availability.

ü  Involved in taking up the Backup, Recovery and maintenance.

ü  Developed MapR Distribution for Apache Hadoop, which speeds up MapReduce jobs with an optimized shuffle algorithm, direct access to the disk, built-in compression, and code written in Java.

ü  Worked on Cisco Tidal Enterprise Scheduler (TES) which is a friendlier alternative to Oozie, the native Hadoop scheduler. Built-in Cisco TES connectors to Hadoop components eliminate manual steps such as writing Sqoop code to download data to HDFS and executing a command to load data to Hive.

ü  Worked on importing and exporting data from Oracle into HDFS and HIVE using Sqoop.

ü  Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.

ü  Created Hive External tables and loaded the data in to tables and query data using HQL.

ü  Wrote shell scripts for rolling day-to-day processes and it is automated.

ü  Collected the logs data from web servers and integrated in to HDFS using Flume.

ü  Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.

ü  Involved in commissioning and decommissioning at the time of node failure.

ü  Implemented Partitioning and Bucketing concepts using Hive.

ü  Involved in upgrading clusters to Cloudera Distributed Clusters and deployed into CDH5.

ü  Taken ad-hoc requests to get the non PII (Personally Identifiable Information) from the production cluster to non-production cluster to test out various scenarios.

Environment: Hadoop, MapReduce, Hive, Cisco Tidal Enterprise Scheduler (TES), Pig, HBase, Sqoop, Flume, ZooKeeper, Cloudera Distributed Hadoop5 (CDH5), Cloudera Manager.

Hadoop Admin                                                             August 2010 to Oct. 2012

IBM - Bangalore, Karnataka

Responsibilities:

ü  Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.

ü  Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.

ü  Expertise with NoSQL databases like HBase, Cassandra, DynamoDB (AWS) and MongoDB.

ü  Involved in migrating java test framework to python flask.

ü  Responsible for developing data pipeline using HD Insight, flume, Sqoop and pig to extract the data from weblogs and store in HDFS.

ü  Experience in working with Ranger in enabling metadata management, governance and audit.

ü  Installed and configured Hadoop, MapReduce, HDFS (Hadoop Distributed File System), developed multiple MapReduce jobs in java for data cleaning.

ü  Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.

ü  Involved in debugging and Tuning the PL/SQL code, tuning queries, optimization for the Oracle, Teradata, and DB2 database.

ü  Experience in methodologies such as Agile, Scrum and Test driven development.

ü  Worked on installing cluster, commissioning & decommissioning of Data Nodes, Name Node recovery, capacity planning, Cassandra and slots configuration.

ü  Installed, configured, and administered a small Hadoop clusters consisting of 10 nodes. Monitored cluster for performance and networking and data integrity issues.

ü  Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing log files.

ü  Architecture and designed Hadoop 30 nodes Innovation Cluster with SQRRL, SPARK, Puppet, HDP.

ü  Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive.

ü  Load log data into HDFS using Flume, Kafka and performing ETL integrations.

ü  Created 25+ Linux Bash scripts for users, groups, data distribution, capacity planning, and system monitoring.

ü  Install OS and administrated Hadoop stack with CDH5 (with YARN) Cloudera Distribution including configuration management, monitoring, debugging, and performance tuning.

ü  Supported MapReduce Programs and distributed applications running on the Hadoop cluster.

ü  Scripting Hadoop package installation and configuration to support fully-automated deployments.

ü  Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters.

Environment: Hive, Pig, HBase, Zookeeper and Sqoop, ETL, Ambari, Linux Cent OS, HBase, MongoDB, Cassandra, Ganglia and Cloudera Manager

Linux Administrator                                                                 May 2008 to July 2010

AES Corporation - Hyderabad, Andhra Pradesh

Responsibilities:

ü  Installing Configuration and maintenance of Solaris 9/10/11, RedHat Linux 4/5/6, SUSE 10.3, 11.1, HP-UX 11.x and IBM AIX operating systems.

ü  Configured NIM in AIX 5.x, 6.x, 7.x, KickStart Server in Linux 4.x, 5.x and JumpStart Server in Solaris 9, 10 u.x and build severs using NIM, kickStart and JumpStart server.

ü  Worked Primarily on RHEL 4/5, HPUX, and Solaris Operating Systems.

ü  Text processing, also in network programming with Perl Scripting.

ü  Worked with installation, configuration, tuning, security, backup, recovery and upgrades of UNIX and Linux (Red Hat, Ubuntu, and SuSE)

ü  Creating, cloning Linux Virtual Machines, templates using VMware Virtual Client 3.5 and migrating servers between ESX hosts.

ü  Involved in migration activities using Redhat LVM, Solaris LVM, Veritas and EMC Open Migrator.

ü  Installation of OAS (Oracle Application Server) on Solaris 9 and its configuration with oracle database.

ü  Writing Shell and Perl Scripting for automation of job.

ü  Tuning the kernel parameters based on the application/database requirement

ü  Used Veritas File system (VxFS) and Veritas Volume Manager (VxVM) to configure RAID 1 and RAID 5 Storage Systems on Sun Solaris.

ü  File system tuning, growing, and shrinking with Veritas File system 3.5/ 4.x.

ü  Installed and configured GFS cluster for holding databases.

ü  Manage user accounts for the team access for Redhat Satellite Server.

ü  Build channels and pull the packages from master Redhat Satellite Server.

ü  Troubleshooting hardware, software and configuration problems for various protocols and topologies.

ü  Installation, Configuration and Administration of Web Logic, WebSphere and Apache Servers in Linux and Solaris Servers.

ü  Configured open LDAP Red Hat Linux systems.

ü  Setup optimal RAID levels (fault tolerance) for protected data storage in NAS environments.

ü  Install and configure DHCP, DNS (BIND, MS), web (Apache, IIS), mail (SMTP IMAP and POP3) and file servers.

ü  Maintaining Remedy environments used for the ticketing system.

ü  Created new slices, mounted new file systems and uncounted file systems.

ü  Expertise in troubleshooting the systems and managing LDAP, DNS, DHCP and NIS.

ü  Worked with different Active directory databases like Microsoft AD, Tivoli Directory server with LDAP.

ü  Worked on making DNS entries to establish connection from server to DB2 database.

ü  Performed patching, backups on multiple environments of Solaris, Linux and VMware.

ü  Assisted other UNIX administrators when help was needed (i.e. creating UNIX accounts, writing scripts to perform system administrator functions, responding to trouble tickets, etc.)

ü  Involved in preparation of functional and system specifications. Estimated storage requirements for applications.  

Environment: Red Hat Linux (RHEL 3/4/5), Solaris 10, Logical Volume Manager, Sun & Veritas Cluster Server, VMWare, Global File System, Redhat Cluster Servers.