The Engineering of Knowledge on Demand

  • Heath Hohwald
Engineering

TL:DR

  1. What's the engineering of knowledge on demand all about? Using a Small Data approach to building and exposing a specialized, data-driven knowledge graph 

  2. What are the unexpected advantages of working with "Small Data"? Achieving a deep understanding of the data, high quality, working directly with the people creating the data, and creatively squeezing value out of data

  3. What are the core soft and hard skills needed for creating a novel data-driven knowledge graph? User focus, business acumen, search skills, UI/ front end wizardry, data architecture

  4. Why join us? Work in a fast-growing industry, enjoy a great team culture, constantly learn and improve your skills, and solve challenging and ambiguous problems

. . .

As we continuously improve our industry-leading knowledge on-demand offer, organizing and structuring information is at the heart of what we do at AlphaSights. On the Data Team, we're search and data specialists. Every day, we work to forward the company's quest to leverage highly specialized information relevant to the knowledge economy. We're out to transform the knowledge industry, one data point at a time. 

What is Knowledge on Demand?

Every day, decision-makers around the globe look to gather more information and better understand different market opportunities. Our challenge is to discover and curate knowledge that's dispersed across millions of minds, doing so at the speed needed for an on-demand offering. To meet our clients' expectations, we need to access, structure, and build a deep understanding of knowledge and its holders. While most of the headlines in the news focus on "Big Data", we've embraced a "Small Data" approach for building a data-rich knowledge graph to accomplish our mission.

Using Small Data to Build Our Knowledge Graph

Can a crack knowledge search team not be all-in on "Big Data?" Interestingly, we think the answer is "Yes!" Our approach to serving as the go-to knowledge partner for our clients is to specialize, not boil the ocean. Many companies are awash in high-volume, low-value data points. The general approach often applied is to try and ingest every last bit of information available out in the digital wild, hoping there are some diamonds in all that hay. Countless cycles are then devoted to trying to manage and mine all the data, most of which might well be garbage. At AlphaSights, we're taking a different approach. We put a laser focus on the thin slice of the world that is most relevant to our clients. We then go miles deep, capturing, structuring, and enriching data that matters with an extremely high bar for quality. We feed the data into our ever-expanding knowledge graph, connecting it more densely and intelligently, making it actionable for our users. Instead of "Big Data", we find that "Small Data" is both up to the task and delivers a rewarding developer experience.

The Unexpected Advantages of Working with Small Data 

Embracing "Small Data" has had some unexpectedly positive consequences for our team. First, we're honest with ourselves: we measure success by the value we generate at the end of the data pipeline, not by how many terabytes of data go into it. Second, we often get to work closely with the data creators, making our work more personable and enjoyable. Rather than treat data as a lifeless string of bits to manipulate, we regularly meet with our users, clients, and product managers. We work to understand what they're really trying to accomplish, rooted in verified business and user needs. Only after reaching a deep understanding do we work to put together data and software requirements, more confident that the end result will delight our users. Third, our "Small Data" approach means we can emphasize quality over quantity. We can understand and influence the full data pipeline, especially the inputs, and we can feel proud of the end results. 

What it Takes to Build a Knowledge Graph

So what does building a knowledge graph look like on the ground? First and foremost, it's about delighting users. We've found that our most successful engineers are the ones that really care about how our offering enables our users and clients. One unexpected benefit to being very user-focused is a tight feedback loop with our users. It can be quite uplifting to receive direct feedback in meetings or through Slack about how awesome a new feature is.

Moving to more tactical skills, it takes quite a variety of abilities to engineer a knowledge graph unlike anything that's ever been built. On the shortlist:

  • Data design: "Small Data" requires smart data design even more than "Big Data" does. To ensure quality and also capture and structure the right data requires good data instincts, a deep understanding of data pipelines, and, when needed, great debugging skills.

  • Architecture: Organizing and managing many disparate pieces of our data-driven puzzle requires top-notch architecture skills. Are we painting ourselves into a corner? Will we be able to accommodate new data fields a year from now? How do we keep these different systems in sync? How do we coordinate multiple queues? 

  • Search skills: Figuring out how to expose and navigate knowledge is quite a challenge! Organizing, indexing, updating, filtering, and backfilling data are at the heart of what we do. This requires deep knowledge of search engines, message queues, APIs, and general backend skills.  

  • UI/ front end/ interactivity: We want the interface to knowledge to be slick, empowering, intuitive, and low-latency. It's a challenge for sure! To pull this off requires great front-end engineers and designers. We lean heavily on React, as well as our in-house design system. We are continually improving our custom component library. 

Why Join AlphaSights?

Of course, nobody is an expert across the board and checks all the boxes above. Our ambitions require a diverse team that enjoys collaboration and continuous self-improvement. We're always looking to bring on new team members. Some particular advantages of joining us:

  • Steep learning curve:  We love developing great engineers. Apart from learning from your teammates, we support a learning culture with regular tech talks, sandbox days for testing out new ideas every 2 weeks, budget for courses and conferences, and more.   

  • Remote work: While we offer a great in-office experience, we also have strong support for full-time remote work, both now and into the future. You can read more about our remote-first team here.

  • Team culture: We have a great team culture for both in-office and remote engineers. We greatly enjoy working with each other — just ask our engineers! Or ask our former engineers, who've gone on to companies like Netflix, Google, etc, and reflect positively on the transferable skills they learned at AlphaSights. This is a great place to work!

Interested in joining us or hearing more? Check out our open roles here.