Engineering Manager (Fullstack), GenAI Safety & Evaluation
Company: Scale AI
Location: San Francisco
Posted on: April 19, 2024
|
|
|
Job Description:
Software is eating the world, but AI is eating software. We live
in unprecedented times - AI has the potential to exponentially
augment human intelligence. As the world adjusts to this new
reality, leading tech companies are racing to build LLMs at billion
scale, while large enterprises figure out how to add it to their
products. To ensure that these models are safe, aligned, and highly
useful, they require extremely high quality human-generated data
and evaluation. Since before the launch of ChatGPT, through to the
latest generation of frontier models coming out today, Scale has
been at the forefront of providing the post-training, fine-tuning,
and human preference alignment (RLHF) data needed to ensure these
models are capable, aligned, and useful via our Generative AI Data
Engine. The data we are producing is some of the most important
work for how humanity will interact with AI.
As customers train their models on this data, and constantly aim to
improve them, a critical need is having trustworthy evaluations of
model performance, and an ability to identify weaknesses and
potential vulnerabilities. Conducting these human evaluations
constitutes a significant and growing portion of Scale's work-and
thus assisting model developers in iteratively understanding where
to focus their technical investments.
The GenAI Safety & Evaluation product team at Scale is at the heart
of this work, building a world-class customer-facing model
evaluation platform. This platform enables customers to easily
launch new evaluation workflows, deep dive into evaluation results
down to the test case level to understand weaknesses and benchmark
performance, and use these insights to drive model development
roadmaps. In building this product, you will have a chance to shape
the way that models across the industry are evaluated, impacting
billions of people around the world. And as a newer product at
Scale, you will have the opportunity to build something impactful
from the ground up.
As part of the T&E team, you will partner closely with
researchers from Scale's on productization of novel research, as
well as Scale's expert red team, which supports AI safety via
rigorous model testing trusted by the , major enterprises, and
leading model developers.
We're looking for Senior Engineering Managers to join our team. You
will be responsible for hiring, training and managing a high
performing engineering team.
You will:
Ideally you'd have:
Nice to haves:
The base salary range for this full-time position in our hub
locations of San Francisco is $212,800 - $255,360. Compensation
packages at Scale include base salary, equity, and benefits. The
range displayed on each job posting reflects the minimum and
maximum target for new hire salaries for the position, determined
by work location and additional factors, including job-related
skills, experience, interview performance, and relevant education
or training. Your recruiter can share more about the specific
salary range for your preferred location during the hiring process.
Scale employees are also granted Stock Options that are awarded
upon board of director approval. You'll also receive benefits
including, but not limited to: Comprehensive health, dental and
vision coverage, retirement benefits, a learning and development
stipend, and generous PTO. Additionally, this role may be eligible
for additional benefits such as a commuter stipend.
#J-18808-Ljbffr
Keywords: Scale AI, Novato , Engineering Manager (Fullstack), GenAI Safety & Evaluation, Executive , San Francisco, California
|
Click
here to apply!
|
