Senior Big-Data Engineer

Full Time Tyne & Wear IT Job Pro UK

Senior Big Data Engineer – Up to £64,656 + up to 27% great pension+ great benefits +performance related bonus + flexible working + work life balance

Do you have expertise in Hadoop stack, Spark, Hive or Impala?

Do you have experience in ETL tooling pipelines?

Do you have proven experience in Cloudera and building big-data capability?

Are you ready to be part of an epic digital transformation?

Then look no further, we'd love to hear from you!

This is not your run of the mill Big Data Engineer role. This is an opportunity to shape solutions and build capability using new and emerging technologies, tools and practices that will change lives.

You'll have a strong grasp of Cloudera solutions in a complex environment and use this expertise to steer in constructively challenging potential issues both on a strategic and technical level.

You'll be inquisitive when building creative solutions and be able to articulate, influence and technically qualify decisions that may be new or never attempted before.

Your approach will be collaborative when presenting information and to ensure that you have a "buy in" on data vision that will help build and enhance the platform.

We're looking for talented Big Data Engineer to join our growing community of experts on our Data & Analytics practice focusing on the Risk and Intelligence service that looks at Fraud & Error as a key part of one of the biggest and highest profile digital transformation programmes in the world.

Using fresh ideas and leading edge technologies – and putting the user at the centre of everything we do – we create innovative digital solutions that make a difference to the lives of our 22 million users.

You and your role

The Department for Work and Pensions (DWP) is one of the largest Government Departments, with an enormously ambitious and challenging agenda, often working in partnership with the private and voluntary sector. We provide services and support to over 20 million people, are responsible for over 730 million benefit payments each year, and manage over £180 billion per year. At the same time, we are leading a once in a generation programme of welfare reform to:

* Understand and tackle the root causes of poverty rather than its symptoms
* Encourage people to work and making work pay
* Encourage disabled people and those with ill health to work and be independent
* Provide a decent income for people of pension age and promoting saving for retirement
* Provide value for money and reducing levels of fraud and error
* Reduce work-related death and serious injury in workplaces through the health and safety executive

The Senior Big Data Engineer role sits right at the heart of that transformation.

Big Data Engineers are responsible for the creation and maintenance of analytics infrastructure, providing the foundations for data needs. You will be responsible for the development, construction, maintenance and testing of architectures, such as databases and large-scale processing systems, implementing data flows to connect operational systems and writing and executing ETL scripts and codes
As a Senior Big Data Engineer you will be responsible for leading, managing and delivering all aspects of data development activities to meet customer needs to agreed timescales within an Agile /Waterfall environment.

You will own the overall Data Engineering Development process within Data Engineering, alongside setting strategies for products and the wider community.

You will also provide technical leadership to a team of Data Engineers, along with providing oversight to the products they develop.

This is a big challenge, but also a big opportunity. You will get the chance to work on one of the biggest, most ambitious Digital transformation programmes in the UK. You will have the chance to use and stretch your abilities, strengthen your experience and continue your professional development in a supportive environment. You will work alongside people who care passionately about public service, making a difference and delivering for our customers.

Essential Technical Skills & Experience

A strong candidate will be able to demonstrate the essential criteria listed below:

Extensive proven experience of big data engineering techniques and concepts using the Hadoop Stack (Cloudera/EMR), including processing and storage using HDFS, Spark, Hive and Impala.
Expert level experience in the development of complex data routines and patterns utilising the Cloudera distribution of Hadoop.
Experience of design and implementation of data storage, including HDFS, relational and NoSQL.
Experience of developing/utilising programming and query languages e.g. SQL, Java, Scala.
Monitoring performance and advising on any required infrastructure or changes.
Prioritise and role-model continuous self-learning and development. Identify areas individuals and teams need to develop in order to achieve future objectivesWhat will I be doing?

As a Senior Big Data Engineer expect to be..

Developing complex data processing pipelines in a cloud or on premise environment in line with build standards and ensure solutions are robust and scalable
Providing technical leadership, setting strategic direction to ensure the products that are developed conform to data standards and strategic principals
Leading, managing and delivering all aspects of code development activities for data ingest routines for unstructured data
Implementing and maintaining coding and tooling standards and ensuring adherence by the wider community
Responsible for leading, managing and delivering all aspects of development activities to meet customer needs and promoting the development of reusable, robust and scalable products e.g. code, citizen centric data models, exploitation products
Work with other technologists, including Lead Data Engineers and Data Architects to investigate and develop solutionsWhere You'll Work

You'll join us in our brilliant digital hub in Newcastle or London.

Find out more here:

We also have all the tech and tools we need to work flexibly. Many of us are working at home at present due to Covid-19.

DWP is the UK's biggest government department with over 80,000 diverse and brilliant people on our team. We are an equal opportunity employer and we welcome applications regardless of age, disability, gender, race or sexuality.

Details. Wages. Perks.

Each of our hubs is a modern, vibrant workplace that was designed for and by members of our digital community.

We provide the latest tech and tooling and foster a collaborative culture to help our agile, multidisciplinary teams think big and try new things. You will work alongside people who care passionately about public service, making a difference and delivering for our customers.

In return for your skills we offer competitive salary up to £64,656 per annum, a brilliant civil service pension of 27% + and a generous leave package.

We also have a broad benefits package built around your work-life balance which includes:

Flexible working
Family friendly policies
Volunteering and charitable giving
Discounts and savings on shopping, fun days out and more
Interest-free loans to buy a bike or a season ticket, so it's even easier for you to get to work and start making a difference
Sports and social activities
Learning and development tailored to your careerBuilding Our Future

We believe the work we do really matters, so we're serious about investing in your ability.

You'll have access to a huge suite of training and learning opportunities to complement the new skills and experience you'll acquire on the job. You'll share ideas with colleagues from across the tech community, and grow your skills via our coaching and mentoring programmes.

For further information click apply or if you require more information before applying please contact

To APPLY just click on the link below

To apply for this job please visit itjobpro.co.uk.