
Raman P Shanthirangam
- Big Data Architect
- Dallas, TX
- Member Since Jun 01, 2023
Raman P Shanthirangam
Summary:
· 22 years of professional experience in building Enterprise applications in US.
· Experience in developing software solutions on Big Data Platforms in Hortonworks, Cloudera and AWS.
· Hands on development experience in Hadoop ecosystem components, Map Reduce, YARN, HDFS, HBase , Apache Hive, Apache Sqoop, Pig, Zookeeper, Oozie, SOLR, Flume
· Experience in cloud computing , Amazon Cloud services , Microsoft Azure , AWS, SaaS, IaaS and PaaS
· Data Lake design and development using Cloudera on AWS (EC2,S3, RDS) , Amazon Redshift
· Expertise in big data application tuning and optimization
· Expertise in Scala, Python , Apache Spark RDD, Spark SQL , Spark Streaming, DataFrames and Kafka
· Familiarity with machine learning pattern using Mahout and Spark MLlib
· Expertise in Business Intelligence tools—Tableau , PowerBI and QlikView
· Expertise in NoSQL Database HBASE, Cassandra , Mongo DB and Apache Phoenix
· Experience in configuring cluster and installing the services and monitoring the cluster.
· Technical Leader with 15 years of experience in HIGH volume database design and architecture
· Expertise in gathering business requirements, capacity planning, resource planning, functional analysis, design, implementation and gap analysis.
· Hands on development and basic administration experience in Oracle 12c/11g/10g/9i/8i/7 RDBS server.
· Expertise in Conceptual, Logical and Physical design of both OLTP and OLAP (Data warehouse) databases.
· Very strong programming skills in SQL, ANSI SQL, PL/SQL, Database Packages/Procedures/Triggers, ERWIN, Oracle SQL Modeler, Oracle Designer , SQL*Loader , Data Pump and Data Base Administration tools.
· Expertise in high transaction database modeling, design and development.
· Sound knowledge in MySQL, Teradata, MS SQL server and ETL Tools (SAS, Data Stage and INFORMATICA).
· Expertise in building J2EE Enterprise applications for financial, medical, automobile, insurance, telemarketing, production planning, MRP II and human resources management industries.
· Experience in complete software development life cycle (SDLC) and software maintenance.
· Very strong programming skills in Java, Servlets, JDBC, JSP, HTML and JavaScript.
· Strong programming skills in EJB, Java/J2EE , Struts framework, Spring Frame Work and Spring MVC
· Sound knowledge in Hibernate, JPA and IBATIS.
· Hands on experience in WebSphere App Server 6.x /7.x, Web Logic 9.x/10.x, RAD 7.x/8.x and WSAD.
· Strong programming skills in Business Objects and Crystal XI.
· Sound knowledge in RUP and Agile methodology- SCRUM, Rational Tools and Eclipse IDE.
· Experience in onshore-off shore round the clock project development and maintenance model.
· Excellent written and verbal communication skills.
Educational Qualification:
· Bachelor of Technology BS (4 Years Full time program )
· Masters in Design MS (2 Years Full Time program)
Technical Skills:
· HDFS, Hive, Sqoop, PIG ,Oozie, Flume, KAFKA, SOLR, Zookeeper, MAP Reduction , YARN , TEZ, HUE, Teradata SQL Assistant for Hadoop, IMPALA, Zeppelin, Ambari, SPARK, NoSQL, Dynamo DB, HBase , MongoDB, Apache Phoenix , Ambari, Cloudera Manager, AWS , EC2, S3, SaaS, IaaS , PaaS , Amazon Redshift , Windows Azure, Oracle Data Base Server (12c/11g/10g/9i/8i/7/6)MySQL, Teradata, MS SQL Server, ANSI SQL, PL/SQL (All versions), TSQL, ERWIN, SQL data modeler, Oracle designer 2000/6i/11g, Oracle SQL developer 3.x, TOAD, SQL Navigator, Oracle JDeveloper, ETL Tools – SAS9.x , Data Stage 8.x,9.x , INFORMATICA 6.x, Java2, Java 5/6/7, SCALA, HTML, XML, UML, FpML, Java Script, JSP, Servlets, EJB 2.x, DTD, XSD, XSLT, AJAX Struts Frame Work, Apache Tiles, Spring Frame Work, Apache Commons, JSON , AVRO, Parquet, Hibernate, Design Patterns, ANT, Hudson, SOAP, WSDL, JAXB , JUnit, Eclipse 3.x,RAD6.x/7.x/8.x WSAD5.x, Change Man, PVCS, Visual Source Safe, Clear Case, IBM Web Sphere (6.x/ 7.x) BEA Web Logic (8.x/9.x,10.x), Tomcat, ,MS Visio, Power Point , MS Project , MS Office products, Developer, Forms, Reports, SQL*Loader, Data Pump, Oracle Financials 10.7, 11i, Tableau, Qlikview, Business Objects, Crystal Enterprise XI, Rational Rose, MS Windows (95/98, NT/XP/Vista/7/10.x),Unix (HP-UX, Sun Solaris & AIX), Linux, PYTHON and Shell scripts
Professional Experience:
AT&T – Dallas
April 2017 – To Date
Big Data Architect, Team Size 8
Responsibilities
· Define project scope, goals and project deliverables interacting with AT&T Product Managers
· Responsible for the Design and Architecture of Hive Data warehouse for 1 PB database
· Heavy coding efforts to develop and test various Hadoop application components.
· Code and Test hundreds of hive programs using Python, hive HQL , Scala and Spark SQL
· Code dozens of SQOOP import to support data ingestion/ETL to AT&T 790 Nodes Hadoop Cluster
· Create dozens of SQOOP export programs for customer interface into Oracle and SQL Server RDBMS
· Expertise in Scala, Spark SQL, Data Frames and Spark RDD
· Expertise in Hortonworks, Beeline, Linux Scripting , Kerberos, Kafka , Zookeeper , Ambari and Zeppelin
· Deploy hive, spark and python programs to PROD environment
· Expertise in HUE, HWX Hive View 2.x , Teradata SQL Assistant and TOAD for Hadoop
· Develop process automation using Oozie and Crontab
· Create BI data visualization components using Microsoft PowerBI
· Hadoop application tuning of Hive and Spark program to optimize the latency and through put
· Interface Hive Data warehouse environment with NoSQL database
· Train and Mentor other team members in the group
· Support Map Reduction programs written SCALA and Hive HQL on the Hadoop cluster
· Tune Queries run on TEZ Hortonworks and Hadoop MR
JP Morgan Chase – Dallas
June 2015 – March 2017
Big Data Architect, Team Size 24
Responsibilities
· Perform data modeling and design for Chase Mortgage, HELOC and Auto loan applications
· Define and manage the architecture and life cycle of Hadoop and SPARK projects
· Develop Map Reduction jobs in Hadoop in Scala/Python/Java/PIG/Hive
· Expertise in writing Scala code to run SPARK jobs in Hadoop HDFS cluster
· Expertise in SPARK Streaming using KAFKA
· Expertise in SPARK SQL, DataFrames , RDD and MLlib
· Experience in writing Pig Latin scripts to load data in to HBase in AVRO format
· Hands on experience in Hive , HiveQL , Impala and Parquet
· Hands on experience in NoSQL Databases Mongo DB and HBase
· Hands on experience with data loading tools like Flume, Sqoop
· Manage Hadoop jobs using scheduler Oozie
· Cluster Coordination services through Zookeeper
· Expertise in Data visualization and Business intelligence using Tableau
· Create database design artifacts, ERD, DFD, conceptual, logical and physical designs using ERWIN for various databases, Oracle , MySQL and NoSQL HBase
· Design data warehouse for star schemas, dimensional, bottom-up Kimball approach
· Design architecture for ETL loads.
· Lead and hands-on effort in Develop, Test and Optimize about 300 Oracle APIs-business logic units in PL/SQL
· BMC Control M designs and implementation for project automation.
· Conversion of SQL Server database procedures into Oracle database packages in PL/SQL using MS visual studio and TOAD.
· Source code control of Oracle APIs, Database packages, Triggers, DDL, DML and DCL scripts in Subversion
· Implementation of projects using Agile SCRUM product development strategy.
· Mentor and train junior staff in the team.
· Demonstrated competence around the areas of product knowledge, relationship building, professionalism and personal excellence, leadership, people focus, and business development.
Northern Trust – Chicago
June 2014 – May 2015
Hadoop Architect/ Solution Architect, Team Size 15
Responsibilities
· Participates in facilitating the definition of project scope, goals and project deliverables
· Responsible for managing the development and deployment of Hadoop application components
· End-to- End Data Lake implementation on AWS, EC2, S3 using Cloudera
· Used Oozie to automate data loading into the Hadoop Distributed File System and PIG to process the data
· Provided design recommendations and thought leadership to upper management that improved processing speed and data storage problems by leaps and bounds
· Hands on experience to move data from RDBMS and data warehouse to Hadoop HDFS cluster using SQOOP
· Data visualization efforts against HDFS , Hive and NoSQL databases Mongo DB and Casandra
· Perform data modeling and analysis for NT Compliance Analyst and Liquidity applications
· Create data modeling artifacts using ERWIN for OLTP and OLAP (Data warehouse) projects
· Create detailed design for new projects. Generate DDL, DML, DCL scripts for both traditional and dimensional
· Create design artifacts and documentation in UML (ERD, DFDs, Sequence diagrams, Class Diagrams…)
· Design and create database tables with partitions to store billions of rows
· Code and Test several database packages with several stored procedures and functions
· Develop and test Data Stage (ETL) Jobs to extract data into data repository and Oracle database from upstream systems
· Written a number of shell scripts and parser routines to automate data extraction from several upstream systems Bloomberg, AIP, NDR , FpML, SWAPs ,Options and OTC DataMart
· Work on project automation using Control M scheduler
· Expertise in Data visualization and Business intelligence using Tableau and Qlikview
Citi Bank (Credit Card Division), Elk Grove Village (Chicago)
June 2012 – May 2014
Technical Lead/ Data Architect, Team Size 21
Responsibilities
· Design Big Data model to transfer data into Hadoop echo systems from Oracle and SaaS data stores as part of company’s new initiate data analytics
· Data visualization efforts against HDFS and Hive
· Written code to move data into HDFS cluster using SQOOP and PIG Latin
· Perform data modeling and analysis for a 5 Terabytes OLTP data model.
· Create database design with integrity constraints and triggers.
· Create design artifacts and documentation for OLTP and Data warehouse.
· Work with business stake holders and business analysts to create credit models.
· Design and create database tables with partitions to store billions of rows.
· Perform basic database administration on DEV and QA instances.
· Develop over 30 database packages with dozens of stored procedures and functions.
· Written several SQL*Loader script to migrate data from legacy databases.
· Written a number of shell scripts to automate data loads from CSV files.
· Write and optimize hundreds of SQL queries and database views for Java/J2EE UI interface.
· Design data warehouse for star schemas, Top-down Inmon methodology.
· Develop and implement data integration architecture with OLAP database using SAS.
· Develop technical detail design for all the major tasks.
· Assign tasks to team members onsite/offshore and monitor status till completion.
· Conduct project status and follow-up meetings and discussions.
· IBM application server WebSphere 7.x administration tasks.
· Perform root cause failure analysis and recommend resolutions.
· Mentor and guide junior team members onsite/offshore.
· Guide QA team to test the software components developed.
· Involved in unit testing, integration testing, feature testing and Regression testing.
· Participate in software gap analysis with business analysts and product managers.
· Manage open tickets and find resolutions.
JP Morgan Chase, Chicago
March 2011 – May 2012
Technical Lead/Solution Architect, Team Size 30
Responsibilities
· Work with business users and business analysts to develop requirements and translate requirements into technical designs.
· Functional analysis, data modeling and database design for Oracle 11g RDMS Databases.
· Data migration from legacy systems and other DBMS Systems.
· Research advanced concepts to develop reusable solutions to recurring problems.
· Work with product managers on capacity planning.
· Participate in software gap analysis with business analysts and product managers.
· Created several DB packages, stored procedures, functions, and triggers.
· Wrote large number of Oracle queries. Heavy coding effort in PL/SQL. Data base administration.
· Analyze functional requirements, Use Cases and technical specifications for J2EE Architecture and design.
· IBM application server WebSphere 6.x administration tasks.
· Preparation of design documents in UML.
· Project Management and sizing.
· Lead and mentor developers in the team
· Implementation of Struts MVC pattern frame work using Struts2.
· Unit testing using Junit.
· Involved in unit testing, integration testing, feature testing and Regression testing.
· UNIX and Shell scripts, Source control management in Subversion.
Navistar, Melrose Park (Chicago)
Dec 2009 – Feb 2011
Team Lead and Solution Architect
Responsibilities
· Conduct sessions/interviews with the business side to secure detailed information pertaining to application and data usage.
· Worked both directly with business users and with business analysts to develop requirements
· and translate requirements into technical design
· Functional analysis, data modeling and database design for Oracle 11g RDMS Databases
· Preparation of design documents in UML
· Weekly communication and reporting of project status risks and issues
· Construct and optimize complex SQL queries
· Heavy coding effort in PL/SQL
· Assign tasks to team members onsite/offshore and monitor status till completion.
· Mentor and guide junior team members onsite/offshore.
· Guide QA team to test the software components developed.
· UI coding efforts in JSP, JavaScript and HTML
· Develop applications using Spring MVC pattern frame work.
· Integration with IBM MQ series
· Used SOA and Web services for importing BOM structure from TEAMCENTER application
· Develop WSDLs, SOAP.
· Heavy hands on programming efforts in control and UI layer.
US Bank, Chicago
March 2008 – Nov 2009
Solution Architect
Responsibilities
· Analyze functional requirements, Use cases and technical specifications.
· Application architecture and design.
· Preparation of design documents in UML.
· Research advanced concepts to develop reusable solutions to recurring problems.
· Construct and optimize complex SQL queries.
· Go to guy in the team for Oracle queries, stored procedures, functions, packages and triggers.
· Implementation of Spring MVC pattern frame work.
· Data management and persistent layer development in open source ORM tool IBATIS.
· Implementation of Dependency Injection using Light weight Spring frame work
· Heavy hands on server side programming efforts in Java.
· UI coding efforts in JSP, AJAX, JavaScript and HTML.
· Extensive coding efforts to develop oracle data base packages and IBATIS mapping files.
· Design and develop SSL encryption and single sign on features of the application.
· Rational tools Clear case and Clear quest for source control and defect tracking.
· Mentor junior programmers in the team.
JP Morgan Chase, Chicago
Feb 2007 – Feb 2008
Application Architect Treasury Services
Responsibilities
· Data migration and data loading programs in Oracle environment
· Conversion of Oracle PL/SQL systems into J2EE Application.
· Written a number a Database packages and triggers in PL/SQL
· <span style="font-size: