Erik Castro

  • Hadoop / Python Developer
  • Tampa, FL
  • Member Since Jun 12, 2023

Candidates About

 

Erik Castro

Summary:

Senior Software Engineer with over 10 years of experience in application architecture, design, development, and deployment in big data technologies and web applications.

Using programming languages such as Python, Go, and Scala; databases such as Apache Spark EMR, Hadoop, Hive, Cloudera Impala, IBM Netezza, PostgreSQL, MySQL, MariaDB, SQLite; messaging technologies such as Redis, RabbitMQ, AWS SQS; platforms such as Amazon AWS, Google Cloud, and Heroku; Frameworks such as Django, Bottle, Flask, and Play. Familiar with multi-threaded, multiprocessing, and highly concurrent and distributed systems and development.

 

Technical Skills

 

Programming Languages:       Python, Java, Go, Scala

CI/CD:                                                AWS CodeBuild, AWS CodePipeline, AWS Lambda

Databases:                                           Netezza, Oracle, PostgreSQL, MariaSQL, MySQL,                                                          MongoDB, Cloudera Impala, SQLite

Operating Systems:                 Unix, Linux, FreeBSD, Windows

APIs:                                                   Amazon EC2, Amazon S3,

                                                            Amazon Simple Queue Service SQS

                                                            REST and JWT

                                                            Boto Python Libraries for AWS

Misc.:                                      Redis Key-Value Stores, Docker

 

Experience

 

VBE INC

Jan 2015 - Present

Hadoop/Python Developer (Remote)

·         Developed AWS CodeBuild/CodePieline/Lambda for 10+ projects for CI/CD

·         Designed and architected Big Data ingestion techniques for Hadoop Clusters.

·         Designed and architected other client or server systems as required.

·         Migration from Data Center to AWS US East region. Migration to AWS S3 Sevices to store Parquet files. Use of AWS Key Management Services (KMS) and AWS DynamoDB to handle credentials storage and usage. Migration to AWS SQS for queuing/scheduling job batches.

 

·         Designed code to build out highly scalable and reliable software.

  • Created Python ETL code to handle 10 terabytes of uncompressed data on a daily basis, achieving scalability to 100 terabytes per day.
  • Tracked/logged event data for ETL code using MariaSQL and SQLite
  • Created data ingestion Python Code using RabbitMQ and Pika Library - code runs on 300 servers
  • Deployed Python code as RPM packages
  • Created Xbox API Client using JSON Web Tokens
  • Created tools to convert JSON to Parquet.
  • Created tools to manage Avro Schemas for different Avro Tools
  • Tested and refined systems throughout the development cycle of the project.
  • Worked closely with design in the development of products.
  • Wrote clear, maintainable, portable code.
  • Participated in code and architecture reviews.
  • Used Agile/Kanban methods to develop software.
  • Technical Environment:  Python, Hadoop, RabbitMQ, Pika, JSON, Arvo, Parquet

 

Bank of America - Charlotte, NC

Nov 2013 – Oct 2014

Senior Python Developer

·         Developed capital markets applications supporting the fixed income and derivative trading businesses supporting middle office P&L needs using Python 2.7

·         Delivered complete spectrum of technology solutions to the capital markets business with the data warehouse being one key component.

·         Used strong object oriented software development methodologies and agile development environment in Python with Oracle databases and NoSQL object databases.

·         Developed software to build Python OOP based processes to load intraday and end of day trading data into a real-time dimensional data warehouse.

·         Exposed to fixed income, interest rate and FX products and their trade life cycle.

·         Built applications upon BoA’s Quartz platform.

·         Technical Environment:  Python, Oracle, NoSQL, Quartz

 

Catalina Marketing - St. Petersburg, FL

Mar 2012 – Nov 2013

Senior Python Developer

·         Developed Python based software systems to target and personalize distribution of coupons, messages, and advertisements to consumers via mobile, on-line, and in-store networks of select retailers.

·         Data Center Migration to AWS US East and US West EC2 instances. Migration from Redis key value stores to AWS DynamoDB. Migration from Celery to AWS SQS services.

·         Established ETL processes and quality assurance techniques to test and develop scalable systems capable of handling over a billion unique consumer shopping identities and purchase histories per week.

·         Conducted business and data research to develop functional requirements for development projects.

·         Developed software to interact with Netezza and Oracle databases to sustain and lift capacity for various growing-in store networks.

·         Developed web based RESTful API’s and queuing strategies to deliver manageable job processing tasks.

·         Deployed several MongoDB instances for website. Created Python scripts to interact with MongoDB using BSON.

·         Deployed MariaSQL/MYSQL databases as alternatives to SQLite databases for Python backend development.

·         Well versed in object oriented development practices and agile development methodologies.

·         Conducted User Acceptance Testing (UAT) and testing procedures to roll out software projects into Production.

·         Created a set of internal applications using Django. These sites monitored server conditions, ETL work, and other processes.

·         Technical Environment: Netezza, Oracle, PostgreSQL, MySQL, MongoDB, Redis, Memcache, NGINX, HTTP, SSL, CSS, HTML, REST, Python, Bottle, cx_Oracle, Psycopg, Celery, Flask, Bottle, Git

 

Lennar Corporation - North American Title Group - Miami, FL

Mar 2005 – Mar 2012

Devops Manager - Python Developer

·         Worked with the title insurance group holding company’s executive management with their policy and claims systems, correct premium and unearned premium calculations, revenue recognition issues, and agent audits.

·         Assisted in completion of National Association of Insurance Commissioners (NAIC) Form 9 Annual Statement Filing.

·         Coordinated the completion of annual data calls in over 25 licensed states with various regulatory agencies.

·         Developed CRM website using Python 2.6 and Web2Py. Transitioned changed from web.py

·         Over 1000 tables, functions, and pricing algorithms updated annually.

·         Maintained a proprietary system developed with Python and PostgreSQL to track all policies and claims.

·         System contains over 1,000,000 policy contracts and several thousand claims.

·         Supported over 750 users.

·         Developed training software for agents and underwriters.

·         Migrated database system from MySQL.

Key Accomplishments:

  • Successfully integrated the accounting/technical functions between two title insurance underwriters upon merging the companies
  • Back tested different premium revenue models for the Florida Office of Insurance
  • Regulation’s study on title insurance premiums and reissue discounts.
  • Developed key SQL queries and stored procedures to comply with various state’s data calls and regulatory requirements.
  • Developed web service to provide quotes in states where rate tables were promulgated by various Departments of Insurance
  • Revenue recognition lags shortened from 120 days to 15 days
  • Direct Premium Written increased from $25 Million in 2005 to $75 Million in 2010.
  • Expanded from 8 states to 27 states.
  • IT processes sped up revenue recognition and scaled for increased revenue and state licenses/regulations.
  • Technical Environment:  Python 2.6, PostgreSQL, SQLite, Mercurial

 

Education

 

Carnegie Mellon University

Pittsburgh, PA

Bachelors of Science, Logic and Computation

Interdisciplinary program in computer science, mathematics, logic, and statistics

 

Affiliations

 

Tampa Bay Python User Group

-Presentation on Graph Databases Using Python and AllegroGraph

-Presentation on Using Python Bottle For Restful Services