Siddhartha Dalal, a professor of professional practice in the Columbia University School of Professional Studies M.S. in Applied Analytics program with a joint appointment as affiliated faculty in the Department of Statistics, applies data science to high-stakes problems in fields ranging from national security to health care.
Dalal was recently awarded a $3.2 million grant from the Intelligence Advanced Research Projects Activity (IARPA) as Co-PI (co-principal investigator) along with PI Professors Vishal Misra and Co-PI Matt Connely for the BENGAL initiative, a new research project aimed at safeguarding sensitive government information.
In the following Q&A, Dalal discusses the mission of BENGAL, how automation could help the government share more information while protecting national security, and how the same preparation and persistence that carry him through research also shape his pursuits outside the classroom, from open-water swims to marathon training and mountain climbing.
Could you explain the goals of the BENGAL initiative and how its work to protect sensitive data in large language models could affect national security and intelligence sharing?
This project matters a great deal from a national security perspective. The government releases a large volume of documents, some of which are classified at different levels of secrecy to protect national security. The important question then becomes: To what extent can these secrets be reconstructed from documents that have already been released, or from other publicly available sources? A related question is to what extent redacted words and sentences themselves can be reconstructed — if they can, that itself constitutes a leakage of information. This project is building technologies and tools that can answer both questions.
The BENGAL initiative focuses on preventing sensitive data extraction and identifying malicious users in large language models. What are some of the biggest technical challenges in distinguishing legitimate use from potentially malicious behavior, and how is your team approaching them?
Adversarial state actors and other malicious users can use publicly available or covertly gathered information to reconstruct government secrets — but we are not trying to identify malicious users or malicious behavior directly. Instead, the goal is to ensure the information itself cannot be reconstructed, regardless of who asks. We approach this in a few ways: checking already released documents for leakage and screening new documents before release to catch material that, combined with what's already public, could reveal protected information.
As large language models become more widely used, protecting sensitive information while still enabling useful information sharing presents a growing challenge. How does the BENGAL initiative approach this balance, and what technical breakthroughs would you hope to see emerge from the project?
This is a critical balancing challenge. One approach is to check whether newly released information would allow reconstruction of matters important to national security. Rather than withholding all document releases, the goal is to flag which specific documents or passages are unsafe to release and let the rest go out as normal. The automation provided will reduce the burden on the government in managing the clearance process, allow the public to have more information and protect national security at the same time.
Between leading research initiatives, teaching and pursuing ambitious personal goals such as running the New York City Marathon, you have a wide range of commitments. Climbing Mount Kilimanjaro and trekking to Everest Base Camp require careful preparation, risk assessment and adaptation to changing conditions. Do you approach complex research initiatives like BENGAL in a similar way, particularly when working on problems where the potential risks are significant but not always fully understood?
My approach to research is deeply mirrored in my personal challenges. I recently swam from Alcatraz to San Francisco through 55-degree waters and strong currents—a feat others deemed impossible given my medical history. I achieved it through meticulous preparation. In my research, I face similar uncertainty; I rarely know if a hypothesis will succeed, but applying that same rigorous, methodical framework significantly improves my odds of overcoming complex problems.
For example, clinicians routinely rely on a 12-lead electrocardiogram (ECG) to diagnose complex cardiac conditions. In a recent paper, we investigated whether this could be achieved using a single lead—the kind now standard in most consumer smartwatches. While intuitively unlikely because a 12-lead system captures far more data, our proprietary AI model proved otherwise. We demonstrated that a single-lead wearable solution predicts left ventricular hypertrophy (LVH)—an independent predictor of cardiovascular morbidity and mortality—nearly as effectively as a full 12-lead ECG, provided it integrates basic demographics: age, gender, and body mass index. This shifts the paradigm of care from clinical facilities to continuous, remote screening, allowing users to capture an ECG on a smartwatch, upload it to our algorithm, and share the results directly with their physician.
Siddhartha Dalal after running the TCS New York City Marathon in 2024
Your professional work involves solving complex problems at the intersection of data science, technology and security, while your personal pursuits push you to test your physical limits. Do you see a common approach to problem-solving across these areas, and has one ever changed the way you think about the other?
They seem far apart, but they aren't: Both require careful preparation and a willingness to take chances, accept failure when it comes and keep striving for success. For example, recently I was working on improving stochastic gradient descent (SGD), a well-known optimization algorithm for AI problems, and after substantial work I came up with a better algorithm but couldn’t beat Adam, another well-known algorithm. I have temporarily conceded defeat and will pursue a better approach in the future. It's the same mindset that got me through my major athletic activities: You don't quit because one attempt didn't work; you regroup and try again.
About the Program
Columbia University’s Master of Science in Applied Analytics prepares students with the practical data and leadership skills to succeed. The program combines in-depth knowledge of data analytics with the leadership, management and communication principles and tactics necessary to impact decision-making across industries and organizational functions.
Learn more about the program here. The program is available full time and part time, online and on campus.
