IBM Advanced Cognitive Platform Data Engineer in SAN FRANCISCO, California

Job Description

Do you think BIG Data is better data? Do you think data lakes are more fun than real lakes? Do you get excited about building new cognitive tools in the Big Data ecosystem? Are you self-motivated and like working on teams? Do you enjoy building scalable platforms on the cloud? This role might be for you!

IBM is seeking Big Data Engineers with technical experience in building scalable distributed software on the cloud that will combine cognitive computing/Watson technology with traditional data engineering/science and apply it at scale to transform enterprise business processes. The ideal candidate will also be able to modify a traditional open-source or other data stack to incorporate cognition. Successful candidates should come from a strong data engineering background and need to have experience with un-structured and structured data and being able to transform and analyze the data using various tools. We are building new platforms, and may need to move our data to new technologies requiring data parity guarantees between new and old feeds.

Required Professional and Technical Expertise

• 10+ years of related work experience in software engineering, architecture and design

• Experienced in designing and delivering large scale distributed software systems, preferably in a large scale global business

• Experienced in service-oriented architecture (SOA) principles

• Experienced in cloud, microservices and Web services standards and best practices

• 5+ years of experience in architecting and developing a highly scalable, distributed system using open source tools and big data technologies: Hadoop, HBase, Spark, Kafka and Kubernetes

• Experience in cloud-based Agile, DevOps experience with PaaS and IaaS, using Big Data batch and streaming tools, SQL, NoSQL, relational database design, DFS, TensorFlow and methods for efficiently retrieving data

• Ability to define, develop, apply and evangelize advanced software patterns, cognitive capabilities and standards to solve complex problems in Cognitive, NL, ML, DL and AI

• Manage risk and build prototypes or POCs where needed to validate technical approach

• Hands-on ability to gather and process raw data at scale, including writing scripts, web scraping, calling APIs, and writing SQL queries

• Hands-on programming in Java, Scala, Python, SQL

• Excellent technical and critical thinking skills

• Excellent verbal and written communication skills

Required Technical and Professional Expertise

• 10+ years of related work experience in software engineering, architecture and design

Preferred Tech and Prof Experience

Master’s or Doctorate preferred

EO Statement

IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.