Candidates About

Seetha Ramayya

SUMMARY:

Over 6+ years’ experience in IT industry as Hadoop Developer including4+ years’ experience on Hadoop Ecosystem i.e.,MapReduce, Hive, Impala, Flume, Sqoop, Oozie, Kafka, Zookeeper and Spark Systems, Scala, Storm and Python Scripting language. 2+ years’ experience in Java including JSP, Junit, Ajax, Struts, Spring, Hibernates, Servlets, Web Services and hands-on experience on Web Technologies.

 

 

Skills Overview:

·         Experience with the Scala, Spark improving the performance and optimization of the existing algorithms in Hadoop usingSpark Context, Spark-SQL, Pair RDD's, Spark YARN.

·         Experience in OLTP and OLAP design, development, testing, implementation and support of enterprise Datawarehouses.

·         Experienced in facilitating streaming data using Kafka and Storm.

·         Extensive experience in business data science project life cycle including Data Acquisition, Data Cleaning, Data Manipulation, Data Validation, Data Mining, Machine Learning Algorithms, and Visualization.

·         Strong knowledge in HIVE and PIG core functionality by using custom User Defined Function's (UDF), User Defined Table-Generating Functions (UDTF) and User Defined Aggregating Functions (UDAF) for Hive.

·         Experience onProductionizing Apache Nifi. for dataflows with significant processing requirements and controlling security of data flow.

·         Experience on Amazon Web Servies(AWS) with variety of services i.e, S3, EMR, Elastic Search(SOLR), EC2.

·         Designed and developed RDD Seeds using Scala and Cascading.Streaming data to Spark streaming using Kafka

·         Experienced in building highly scalable Big-data solutions using Hadoop distributed platforms i.e., Cloudera.

·         Exposure to Hadoop Distributed Platforms i.e., Hortonworks and MapR.

·         Good understanding of NoSQL databases and hands on work experience in writing applications No SQL Databases HBase, Cassandra and MongoDB.

·         Experienced in installation, configuration, supporting and managing Hadoop Clusters using ApacheCloudera distributions, Hortonworks, Cloud Storage and Amazon web services (AWS).

·         Experience in deploying NiFi Data flow in Production team and Integrating data from multiple sources like Cassandra, MongoDB.

·         Deploying templates to environments can be done via NiFiRestAPI integrated with other automation tools

·         Experienced in Python programming, wrote WebCrawlers using Python.

·         Experienced in developing MapReduce jobs using Scala in Spark-Shell.

·         Good experienced on moving the data in and out of Hadoop RDBMS, No-SQL and UNIX from various systems using SQOOP and other traditional data movement technologies.

·         Experience on Integration of Quartz scheduler with Oozie work flows to get data from multiple data sources in parallel using fork.

·         Good knowledge on tuning the Spark jobs by changing the configuration properties and using broadcastvariables.

·         Developed REST APIs using Java, Play framework and Akka. 

·         Expertise in search technology's like SOLR.

·         Experienced in Stormbuilder topologies to perform cleansing operations before moving data into HBase.

·         Hands on experience in configuring and working with Flume to load the data from multiple sources directly into Hdfs.

·         Experience on configuring fully the Flume agent, suitable for all type of logger data and store them in Avro Sink in Parquet file format and developing 2-tier architecture connecting channels between Avro sinks and Source. 

·         Experience creating Visual report, Graphical analysis and Dashboard reports using Tableau, Informaticaof historical data saved in Hdfsand data analysis using Splunk enterprise edition.

·         Good experience in utilizing Cloud Storage Services like Git. Extensive knowledge in using GitHub.

·         Experienced in job scheduling and monitoring using Oozie, Zookeeper.

·         Experience in Object Oriented concepts, Multithreading and Java/Scala

 

Bigdata Ecosystem	HDFS and Map Reduce, Pig, Hive, Impala, YARN, Oozie, Zookeeper, Apache Spark, Apache NiFi, Apache STORM, Apache Kafka, Sqoop, Flume.
NoSQL Databases	HBase, MongoDB, Cassandra.
Java Technologies	Java, J2EE, JDK1.4/1.5/1.6/1.7/1.8, JDBC, Hibernate, XML, Parsers, JSP 1.2/2, Servlets, EJB, JMS, Struts, Spring Framework, Java Beans, AJAX, JNDI.
Frameworks	MVC, Struts, Hibernate, Spring Framework, Spring Boot.
Databases	Netezza, SQL Server, MySQL, ORACLE, DB2.
Programming Languages	C, C++, Java, J2EE, JDBC, JUnit, Log4j, C#, Python, Scala, Swift, Android, PL/SQL, HQL, Unix, Shell Scripting.
Scripting Languages	Python, Perl, Shell, Sheme, Tcl, Unix Shell Scripts, Windows Power Shell
Web Technologies	HTML, JavaScript, JQuery, Ajax, Boot Strap, Angular JS, Node.js.
Development Methodologies	Waterfall, UML, Design Pattern (Core Java and J2EE), Agile Methodologies (Scrum).
IDE Development Tools	Eclipse, Net Beans, Visual Studio, XCode, Android Studio, Intellij, Jetbrains.
Operating Systems	Windows, Linux, Unix, Ubuntu.
Management Tech	SVN, Git, Jira, Maven.
Web Services	SOAP, RESTFUL API, WSDL.

Client: State Farm, Raleigh, NC
Role: Hadoop/Spark Developer                                                                   Jan,2016-Present

Project Description: The Main theme of this project is to develop a scheme that analyses the policies per day which are valid and which are invalid. We used Flume and Sqoop to store the data in HDFS as XML files and imported into Scala program using Scala libraries and used Hive to process the data into a dataset and processed through a spark SQL and then processed data is transferred to another department using python script.

Responsibilities:

·         Involved in loading data from LINUX file system to HDFS.

·         ImplementedSpark usingScala and utilizing Data frames andSpark SQLAPI, Data Frames and Pair RDD's for faster processing of data and created RDD's, Data Frames and datasets.

·         Used SparkAPI over Cloudera Hadoop YARN to perform analytics on data in Hive.

·         Load the data into SparkRDD and performed in-memory data computation to get faster output response and implemented sparkSQL queries on data formats like Text file, CSV file and XML files.

·         Responsible for gathering requirements, process workflow, data modelling, architecture and design and led application development using Scrum.

·         Worked on large-scale Hadoop YARN cluster for distributed data processing and analysis using Data Bricks Connectors, Spark core, Spark SQL, Sqoop, Pig, Hive, Impala and NoSQL databases.

·         Integrated Maven build and designed workflows to automate the build and deploy process.

·         Used slick to query and storing in database in a Scala fashion using the powerful Scala collectionframework.

·         Collected and aggregated large amounts of web log data from different sources such as webserversin the form of XMLusing ApacheFlume and stored the data into HDFS for analysis.

·         Used Scala libraries to process XML data that was stored in HDFS and processed data was stored in HDFS.

·         Used Spark for interactive queries, processing of streaming data and integration with popular NoSQL database for huge volume of data. 

·         Used HUE for running Hive queries. Created Partitionsper day using Hive to improve performance.

·         Prepared an ETL pipeline with the help of Sqoop, pig and hive to be able to frequently bring in data from the source and make it available for consumption.

·         Involved in developing and designing POC's using Scala, Spark SQL and MLlib libraries then deployed on the Yarn cluster, compared the performance of Spark with Hive and SQL/Oracle.

·         Developed the batch scripts to fetch the data from AWS S3 storage and do required transformations in Scala using Spark framework.

·         Used Kafka for log accumulation like gathering physical log documents off servers and places them in a focal spot like HDFS for handling.

·         Used Spark for interactive queries, processing of streaming data and integration with popular NoSQL database for huge volume of data. 

·         Worked with various HDFS file formats like Avro, Sequence File and various compression formats like Snappy, bzip2.

·         Developed UDF's to pre-process the data and compute various metrics for reporting in both pig and hive.

·         Responsible for implementing Machine learning algorithms like K-Means clustering and collaborative filtering in Spark.

·         Spark streaming collects the data from Kafka in near real time and performs necessary transformations and aggregations on the fly to build the common learner data model.

·         Implemented Spark Scripts using Scala, Spark SQL to access hive tables into spark for faster processing of data.

Environment: Cloudera, HDFS, Spark, Hive, Pig, Sqoop, Putty, HaaS (Hadoop as a Service),Apache Kafka and the AWS, Spark, SPARK SQL, Maven, Java, Scala, SQL and Linux, YARN, Agile Methodology.

Client: BlueCross Blue Shield, Chicago, IL
Role: Hadoop Developer                                                      Aug,2014- Dec,2015 

Project Description: The main intention of this project is to build a unified database from the basic repository hub of Global New Accounts. The basic repository hub contains the transactions details data and logs from different applications and will be in various formats of all the credit card transactions. We are responsible for creating HDFS for audit and use the map reduce and Hive concepts to transform the data. The transformed data will be useful to increase credit card business by using various measures.

Responsibilities:

Ø  Developed Hive (version 0.12.0) scripts to analyze data and mobile numbers are categorized into different segments and promotions are offered to customer based on segments.

Ø  Prepared an ETL pipeline with the help of Sqoop, pig and hive to be able to frequently bring in data from the source and make it available for consumption.

Ø  Processed data into HDFS by developing solutions and analyzed the data using Map Reduce, PIG, and Hive to produce summary results from Hadoop to downstream systems. 

Ø  Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hive and Sqoop. Responsible for building scalable distributed data solutions using Hadoop.

Ø  Used Oozie workflow engine to manage independent Hadoop jobs and to automate several types of Hadoop such as javaMapReduce, Hive and Sqoop as well as system specific jobs.

Ø  Developed Hive (version 0.12.0) scripts to analyze data and mobile numbers are categorized into different segments and promotions are offered to customer based on segments.

Ø  Developed ETL scripts based on technical specifications/Data design documents.

Ø  Involved in loading and transforming large sets of Structured, Semi-Structured and Unstructured data and analyzed them by running Hive queries and Pig scripts.

Ø  Reporting the data to analysts for further tracking of trends per various consumers.

Ø  Used Kafka Streams to Configure Sparkstreaming to get information and then store it in HDFS.

Ø  Wrote MapReduce jobs to generate reports for the number of activities created on a day, during a dumped from the multiple sources and the output was written back to HDFS.

Ø  Involved in using ORC, Avro, Parquet, RCFile and JSON file formats and developed UDFs using Hive, Pig and Sqoop to import files into Hadoop.

Ø  Used Spark-SQL to Load JSON data and create SchemaRDD and loaded it into Hive Tables and handled Structured data using SparkSQL.

Ø  Worked with ApacheSOLR to implement indexing and wrote Custom SOLR query segments to optimize the search.

Ø  Expert knowledge on MongoDBNoSQL data modeling, tuning, disaster recovery and backup.

Ø  Used Zookeeper to manage coordination among the clusters.

Ø  Developed Custom InputFormat, Record Reader, Mapper, Reducer, Partitioner as part of developing end to end Hadoop applications.

Ø  Followed Agile-Scrum project development methodology for implementation of projects, part of the daily scrum meetings and sprint meetings.

Ø  Worked with NoSQL databases like HBase in making HBase tables to load expansive arrangements of semi structured data.

Ø  Assisted in Cluster maintenance, Cluster Monitoring and Troubleshooting, Manage and review data backups and log files.

ENVIRONMENT: Hortonworks, HDFS, Map Reduce, Pig, Mesos, AWS Hive, Sqoop, Scala, Flume, Mahout, HBase, Spark, SPARK SQL, Yarn, Java, Maven, Git, Cloudera, MongoDB, Eclipse and Shell Scripting.

Client: Radiumone, SanFrancisco, CA
Role: J2EE/Hadoop Developer                                            Nov,2012-June,2014

Project Description: This project involves setting up a data repository and use it for search processing for analytical and research purposes. This application provides the capability for large batch processing using Hadoop map reduce jobs using JAVA runtime environment as well as real time search capabilities using Solr cloud environment.

Responsibilities:

Ø  Responsible for gathering business and functional requirements for the development and support of in-house and vendor developed applications.

Ø  Played key role in design and development of new application using J2EE, Servlets, and Spring technologies/frameworks using Service Oriented Architecture (SOA).

Ø  Wrote Action classes, Request Processor, Business Delegate, Business Objects, Service classes and JSP pages.

Ø  Developed validation using Spring's Validation Interface and used Spring Core and MVC develop the applications and access data.

Ø  Design and developed different PL/SQL blocks, Stored Procedures in DB2 database.

Ø  Developed data mapping to create a communication bridge between various application interfaces using XML and XSL.

Ø  Developed the application under JEE architecture, developed, designed dynamic and browser compatible user interfaces using JSP, Custom Tags, HTML, CSS, and JavaScript

Ø  Worked with structured and semi structured data of approximately 100TB with replication factor of 3.

Ø  Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.

Ø  Did various performance optimizations like using distributed cache for small datasets, Partition, Bucketing in hive and Map Side joins.

Ø  Extensively used Hive/HQL or Hive queries to query or search for a string in Hive tables in HDFS.

Ø  Used Flume extensively in gathering and moving log data files from Application Servers to a central location in Hadoop Distributed File System (HDFS).

Ø  Involved in Analyzing Cassandra database and compare it with other open-source NoSQL databases to find which one of them better suites the current requirements.

Ø  Created HBase tables to store various data formats of data coming from different portfolios.

Ø  Involved in complete Implementation lifecycle, specialized in writing custom MapReduce, Pig and Hive programs.

Ø  Worked with Apache SOLR to implement indexing and wrote Custom SOLR query segments to optimize the search.

 

Environment: Java, UNIX, HDFS, Pig, Hive, Spark, Scala, MapReduce, Flume, Sqoop, HBase, Cassandra, Cloudera Distribution, Yarn, Shell scripting, Spring MVC, Oracle 11g J2EE, Java, JDBC, Servlets, JSP, XML, Design Patterns, CSS, HTML, JavaScript 1.2.

Client: SriKari Impetus, Hyderabad, INDIA
Role: Java Developer                                                                        Dec,2011- Nov,2012

Responsibilities:

Ø  Actively involved in the analysis, definition, design, implementation and deployment of full Software Development Life Cycle (SDLC) of the project.

Ø  Used Hibernate, Object Relational Mapping (ORM) solution, technique of mapping data representation from MVC model to Oracle Relational data model with an SQL-basedschema.

Ø  Implemented RESTful web services using Jersey for JAX-RS

Ø  Designed and implemented application using JSP, Spring MVC, JNDI, Spring IOC, Spring Annotations, Spring AOP, Spring Transactions, Hibernate, JDBC, SQL, ANT, JMS, Oracle.

Ø  Used object storage container to store the secured files, and retrieved from API by using Amazon Web Services (AWS).

Ø  Developed various UML diagrams like usecases, class diagrams, interaction diagrams (sequence and collaboration) and activity diagrams.

Ø  Used Multithreading (Concurrent) in programming to improve overall performance using Singleton design pattern in Hibernate Utility class.

Ø  Implemented SOA architecture with Web Services using SOAP, WSDL, UDDI and XML using Apache CXF framework tool/Apache Commons. Worked on parsing the XML files using DOM/SAX parsers.

Ø  Involved in Bug fixing of various modules that were raised by the testing teams in the application during the Integration testing phase.

Ø  Used Junit framework for unit testing of application and Log4j to capture the log that includes runtime exceptions. Used CVS for version control for implementing the application.

ENVIRONMENT: Java, JSP, HTML, CSS, Ubuntu Operating System, JavaScript, AJAX, Servlets, Struts, Hibernate, EJB (Session Beans), Log4J, WebSphere, UML, JNDI, Oracle, Windows XP, LINUX, ANT, Eclipse.