IBM Big Data Engineer/Architect in SAN JOSE, California

Do you think BIG Data is better data? Do you think data lakes are more fun than real lakes? Do you get excited about learning new tools in the BIG Data ecosystem? Are you self-motivated and like working on teams? Do you enjoy building scalable platforms on the cloud? This role might be for you!IBM is seeking Big Data Engineers with technical experience in building scalable distributed software on the cloud that will combine cognitive computing/Watson technology with traditional data engineering/science and apply it at scale to transform enterprise business processes. The ideal candidate will also be able to modify a traditional open-source or other data stack to incorporate cognition. Successful candidates should come from a strong data engineering background and need to have experience with un-structured and structured data and being able to transform and analyze the data using various tools. We are building new platforms, and may need to move our data to new technologies requiring data parity guarantees between new and old feeds.The ideal candidate will have significant experience with much of the following or equivalent:

  • 3+ years’ technical and critical thinking skills

  • 3+ years’ experience of service oriented architecture (SOA) principles and Web services standards and best practices

  • Capability to architect highly scalable distributed systems using open source tools and big data technologies such as Hadoop, HBase, Spark, Storm, etc

  • 3+ years’ experience as a software architect designing and delivering large scale distributed software systems, preferably in large scale global business

  • Experience in cloud-based agile, devops experience with PaaS & IaaS

  • Experience using Big Data batch and streaming tools

  • Ability to define, apply and evangelize advanced software patterns and standards to solve complex problems

  • Experience with SQL, NoSQL, relational database design, and methods for efficiently retrieving data

  • Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)

  • Experience programming in Java, Scala, Python, SQL

  • Build prototypes or POCs where needed, to validate technical approach

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

  • 3+ years of related work experience: in software engineering, architecture and design-designing and delivering large scale distributed software systems, preferably in large scale global business, service oriented architecture (SOA) principles and Web services standards and best practices

  • Hands-on experience as a software architect designing and delivering large scale distributed software systems, preferably in large scale global business

  • Capability to architect highly scalable distributed systems using open source tools and big data technologies such as Hadoop, HBase, Spark, Storm, Etc

  • Experience in: cloud-based agile, devops experience with PaaS & IaaS, using Big Data batch and streaming tools,SQL, NoSQL, relational database design, and methods for efficiently retrieving data

  • Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)

  • Programming in Java, Scala, Python, SQL

IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.