Big Data Specialist

• Based on assignment, requires the technical ability to work in at least one data discipline including data design, data movement, data infrastructure platforms, and data analytics
• Provides data design/data movement services to help develop solutions that improve overall quality of data and support enterprise data strategies
• Uses applied analytics to enable data driven decisions by applying innovative analytic techniques, and providing exploratory research and analysis
• Develops data access related to storing, retrieving or acting on housed data
• Ensures work and/or the work of their teams are in compliance with enterprise policies and policies and standards within their domain of expertise and area of responsibilities
• Design and execute the policies, practices, and procedures that support data quality, data security, data governance, data standardization & integration
• Translates functional and technical requirements into detailed design
• Develops for technical data platforms such as DB2, SQL, IMS across multiple operating systems
• Defines, designs, and documents conceptual and logical data models
• Uses Informatica toolset to develop applications and deploy data integration/data quality applications
• Tests data requirements for the movement, replication, synchronization and validation of data
• Provides support for data movement solutions including extract, transformation, load (ETL)
• Understands data infrastructure needed to support data services and automation
• Proactively increase knowledge in the data industry, its related technologies and tools in order to work on a wide variety of assignments and products
• Provides product and/or process subject matter expertise necessary to support design, development, testing and execution of technology solutions
• Exhibits DevOps mindset where team is accountable for product from inception to sunset

Additional Details:

State farm is seeking a talented Big Data specialist /Engineer who is passionate about everything data to enable our infrastructure team to gain efficiencies in supporting the Big Data Platform as a Service. You will be a member of a team that supports a state of the art Big data platform comprised of multiple clusters and users and will help by working with data for trouble shooting issues, querying logs, develop insights on business use of the platform, query data in search of curious issues impacting various Hadoop clusters and create various data structures that would allow our business partners to effectively use the platform. You would be responsible for working on the architecture team with the Architecture leadership to develop software data components iteratively. You will collaborate with other team members and with Architecture leadership to understand issues and go about suggesting enhancements creating/ enhancing features and writing stories that demonstrate use of data structures and supporting data tools to enhance the operational efficiency of the platform. Your work will be driven by understanding current and historical issues and providing solutions as well as leveraging the backlog list as a source for platform improvement. Making use of innovative techniques to deliver solutions is the heart of who we are seeking. We will be looking for your expertise and ability to question the status quo, ask WHY and provide a solid supporting because and demonstrate the how. We are looking for a team player who is highly engaged, passionate about architecture and delivery of a Hadoop Platform, skill wise someone who is inquisitive about data and will ask probing questions to understand challenging technical issues. Your work will be spread around creating or enhancing various business and technical capabilities that are supported by the platform to enhance the platform and remove obstacles for use.

Who we are: Our team is the Big data platform as a service that manages the infrastructure and enable the big data platform for our LOB’s to leverage for business and technical insights. We manage large data stores that house different types of data such as customer, customer interactions data, Billing and Payments, Policy data to name a few. Our big data platform serves as a foundation for Guided and Operational analytics, machine learning and contextual call routing. Our platform is also being used as a foundation to develop various machine learning algorithms and deploy to the cloud.

What you get: You will get an opportunity to work and learn a variety of technologies and frameworks that are part of the Hadoop stack and suggest various open source tools that could be used to increment the efficiency of the platform. You will get to run individual Proof of concepts to demonstrate capability with full autonomy to take a problem and deliver a solution. The ideal candidate will be experienced with the Hadoop ecosystem, proficient with usage patterns, knowledgeable of the internal working of various ecosystem tools, have a good handle on the industry trends in Hadoop and eagerness to apply innovative software development patterns and code to infrastructure and service support in the Hadoop tool stack. Your work may involve identifying various bottleneck with other infrastructure items that interact with the Hadoop ecosystem such as Networking/security, firewall and so on. You have an direct opportunity to be more than a contributor to code, interact and solve real business issues, gain a good exposure to leadership and line of sight to the next level. You have the flexibility to contribute to solutions that could be open sourced within the State farm community.

Required Skills:

• At least 3-5 years of industry experience in the following:
• Hadoop Data platform broad experience (Solr search & indexing, HBase, HDFS); Data Streaming (Kafka, Flume)
• Working with NO SQL, creation of SOLR collections, reading hdfs files, experience with PUB/SUB
• Data experience with schemas, columnar that includes NOSQL as well as serializing patterns such as AVRO/parquet
• Experience using one or more of the tools such as SCP, DistCp, Informatica and FTP for data movement in and out of Hadoop
• Experience with designing, creation, loading, partitioning and querying HIVE and/or Impala data stores using SQL
• Experience working with Linux/Apache/Tomcat and other open source technologies
• Experience with CI/CD and use within the Hadoop Platform
• Experience with innovative data use within the platform such as but not limited to: Querying Hadoop logs, SPLUNK logs and providing feedback on cluster usage/issues
• Test-driven design
• Agile

Desired Skills:

• Experience with Linux Scripting
• Experience in developing and deploying in a cloud environment including:
• Big Data development (Spark/Python)
• UrbanCode

This position will have an 18-month incumbency period, beginning on the effective date of the position, which must be met before the employee can post for any other lateral State Farm position. Employees may be considered for openings for Technology Engineer positions or promotional opportunities into leadership at any time, and into a promotional position within the same job structure after 6 months from the effective date of the position. In these cases, the remaining incumbency would be waived. The incumbency period does not affect the at-will relationship between State Farm and the employee and does not create an employment contract, nor contractual rights.

*****Applicants are required to be eligible to lawfully work in the U.S. immediately; employer will not sponsor applicants for U.S. work authorization (such as an H-1B visa) for this opportunity***********

Location Details

Tempe, AZ 85281