Monica Sin-Ling Lam is an American computer scientist. She is a professor in the Computer Science Department at Stanford University. == Education == Monica Lam received a B.Sc. from University of British Columbia in 1980 and a Ph.D. in computer science from Carnegie Mellon University in 1987. == Career == Lam joined the faculty of Computer Science at Stanford University in 1988. She has contributed to the research of a wide range of computer systems topics including compilers, program analysis, operating systems, security, computer architecture, and high-performance computing. More recently, she is working in natural language processing, and virtual assistants with an emphasis on privacy protection. She is the faculty director of the Open Virtual Assistant Lab, which organized the first workshop for the World Wide Voice Web. The lab developed the open-source Almond voice assistant, which is sponsored by the National Science Foundation. Almond received Popular Science's Best of What's New award in 2019. Previously, Lam led the SUIF (Stanford University Intermediate Format) Compiler project, which produced a widely used compiler infrastructure known for its locality optimizations and interprocedural parallelization. Many of the compiler techniques she developed have been adopted by industry. Her other research projects included the architecture and compiler for the CMU Warp machine, a systolic array of VLIW processors, and the Stanford DASH distributed shared memory machine. In 1998, she took a sabbatical leave from Stanford to help start Tensilica Inc., a company that specializes in configurable processor cores. In another research project, her program analysis group developed a collection of tools for improving software security and reliability. They developed the first scalable context-sensitive inclusion-based pointer analysis and a freely available tool called BDDBDDB, that allows programmers to express context-sensitive analyses simply by writing Datalog queries. Other tools developed include Griffin, static and dynamic analysis for finding security vulnerabilities in Web applications such as SQL injection, a static and dynamic program query language called QL, a static memory leak detector called Clouseau, a dynamic buffer overrun detector called CRED, and a dynamic error diagnosis tool called DIDUCE. In the Collective project, her research group and she developed the concept of a livePC: subscribers of the livePC will automatically run the latest of the published PC virtual images with each reboot. This approach allows computers to be managed scalably and securely. In 2005, the group started a company called MokaFive to transfer the technology to industry. She also directed the MobiSocial laboratory at Stanford, as part of the Programmable Open Mobile Internet 2020 initiative. Lam is also the cofounder of Omlet, which launched in 2014. Omlet is the first product from MobiSocial. Omlet is an open, decentralized social networking tool, based on an extensible chat platform. Lam chaired the ACM SIGPLAN Programming Languages Design and Implementation Conference in 2000, served on the Editorial Board of ACM Transactions on Computer Systems and numerous program committees for conferences on languages and compilers (PLDI, POPL), operating systems (SOSP), and computer architecture (ASPLOS, ISCA). == Awards and honors == National Academy of Engineering member, 2019 University of British Columbia Computer Science 50th Anniversary Research Award, 2018 Fellow of the ACM, 2007 ACM Programming Language Design and Implementation Best Paper Award in 2004 ACM SIGSOFT Distinguished Paper Award in 2002 ACM Most Influential Programming Language Design and Implementation Paper Award in 2001 NSF Young Investigator award in 1992 Two of her papers were recognized in "20 Years of PLDI--a Selection (1979-1999)" One of her papers was recognized in the "25 Years of the International Symposia on Computer Architecture", 1988. == Selected works == Compilers: Principles, Techniques and Tools (2d Ed) (2006) (the "Dragon Book") by Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman (ISBN 0-321-48681-1) A Systolic Array Optimizing Compiler (1989) (ISBN 0-89838-300-5) Monica Lam, Dissertation
Corpus of Linguistic Acceptability
Corpus of Linguistic Acceptability (CoLA) is a dataset the primary purpose of which is to serve as a benchmark for evaluating the ability of artificial neural networks, including large language models, to judge the grammatical correctness of sentences. It consists of 10,657 English sentences from published linguistics literature that were manually labeled either as grammatical or ungrammatical. == Public version == The publicly available version of CoLA contains 9,594 sentences that belong to training and development sets. It excludes 1,063 sentences reserved for a held-out test set.
Project Debater
Project Debater is an IBM artificial intelligence project, designed to participate in a full live debate with expert human debaters. It follows on from the Watson project which played Jeopardy! == Development == Project Debater was developed at IBM's lab in Haifa, Israel. The project was proposed by Noam Slonim in 2011 as the IBM Research next Grand Challenge, following Deep Blue and the victory of Watson in Jeopardy! It was exposed for the first time in a closed media event at June 18, 2018, in San Francisco, under the leadership of Ranit Aharonov and Slonim, both from the IBM Research lab in Haifa, Israel. The AI technology debated two human debaters, Noa Ovadia, who was the 2016 Israeli debate champion and Dan Zafrir. The two debated on the topics "We should subsidize space exploration" and "Should we increase the use of telemedicine." A demonstration of Project Debater also aired on the Discovery Channel in June 2018 debating the question of whether sports gambling should be legalized. == Live Debate == On February 11, 2019, Project Debater was revealed to the world in a live debate in San Francisco. Nonpartisan media group Intelligence Squared U.S. Debates hosted the debate which was moderated by journalist John Donvan. The debate took place between Project Debater and Harish Natarajan, who holds the world record in number of debate competition victories. The motion was “We should subsidize preschools.” == That's Debatable Television Show == Project Debater was featured in a television series called “That’s Debatable” presented by Intelligence Squared U.S. Debates and Bloomberg Media. For each episode of “That’s Debatable,” Project Debater provided insight into three distinct debate topics on the redistribution of wealth, modern monetary theory, and a US-China space race. More than 5,000 arguments were submitted online from around the world across the three topics, which were then analyzed and distilled into key points that were highlighted on the television show and discussed by human debaters. == Artificial Intelligence Capabilities == To develop Project Debater, the IBM Research team had to endow the system with the following AI capabilities: Data-driven speech writing and delivery: Project Debater is the first demonstration of a computer that can digest massive corpora, and given a short description of a controversial topic, write a well-structured speech, and deliver it with clarity and purpose, while even incorporating humor where appropriate. Listening comprehension: the ability to identify the key concepts and claims hidden within long continuous spoken language. Four minutes of persuasive speech: the guarantee of producing four minutes of persuasive speech. Modeling human dilemmas: modeling the world of human controversy and dilemmas in a unique knowledge representation, enabling the system to suggest principled arguments as needed. An article on the project was published in Nature in March 2021.
CADE ATP System Competition
The CADE ATP System Competition (CASC) is an annual competition of fully automated theorem provers for classical logic. == Competition == CASC is associated with the Conference on Automated Deduction and the International Joint Conference on Automated Reasoning organized by the Association for Automated Reasoning. It has inspired similar competition in related fields, in particular the successful SMT-COMP competition for satisfiability modulo theories, the SAT Competition for propositional reasoners, and the modal logic reasoning competition. The first CASC, CASC-13, was held as part of the 13th Conference on Automated Deduction at Rutgers University, New Brunswick, NJ, in 1996. Among the systems competing were Otter and SETHEO.
On a Red Station, Drifting
On a Red Station, Drifting is a 2012 science fiction novella by Aliette de Bodard. Set in her Xuya Universe, it focuses on two women aboard a space station with a failing artificial intelligence. It received critical acclaim, becoming a finalist for the 2012 Nebula Award for Best Novella, the 2013 Hugo Award for Best Novella, and the 2013 Locus Award for Best Novella. == Plot == Lê Thi Linh is a magistrate of the Dai Viet Empire who is forced to flee her planet after criticizing the Emperor’s wartime policies. At the same time, rebel groups seize control of her planet and kill most of her subordinates. Linh seeks refuge with her distant relatives on Prosper Station. Prosper is controlled by an artificial intelligence called the Honoured Ancestress. Lê Thi Quyen, Linh’s cousin by marriage, manages the day-to-day operations of Prosper while her husband is away at war. Quyen and Linh immediately fall into conflict. Quyen’s brother-in-law Huu Hieu sells his mem-implants, which are copies of their ancestors’ consciousnesses. Meanwhile, the Honoured Ancestress experiences increasingly severe technical problems. Hieu and Linh become close. Hieu plans use the money from the sale of the implants to leave Prosper and marry his lover on a different station. Linh is upset knowing that she will never be able to leave. A visiting cousin, Lady Oahn, provides schematics for the repair of the Honoured Ancestress. In an effort to hurt Quyen, Linh writes an unflattering poem at a banquet honoring Oanh. In doing so, she reveals that Hieu is trying to leave Prosper. Hieu attempts suicide out of shame, but Linh rescues him. Quyen is able to repair the Honoured Ancestress, restoring her functionality at the expense of erasing many of her memories. The Emperor’s Embroidered Guard arrives at Prosper Station in search of Linh. Linh finds the missing mem-implants and returns them to Quyen. Quyen and Linh briefly reconcile before Linh is arrested and removed from Prosper Station. == Major themes == A review in Kirkus wrote that the novel's "familiar setting" was a "departure point" for the novel to explore its themes. The novel explores family ties; almost everyone on Prosper Station is related in some fashion. Additionally, the use of ancestors' mem-implants further explores the concept of family ties, with some descendants being considered more "worthy" than others due to their higher number of implants. The novel also explores questions of worth, as those who fail at ability tests are often forced to become the "lesser partners" in marriages and are discriminated against due to their perceived lack of achievement. The author notes that it is interesting that gender plays no role in the question of worth, and that the majority of the men in the story are actually the "lesser partner" in their marriage. == Style == The novel is divided into three sections. Liz Bourke wrote that each section builds thematically "towards an emotional crescendo". == Reception == Writing for Locus, Liz Bourke praised the novel's exploration of interpersonal conflict between Linh and Quyen, writing that "essentially subverts the popularly-understood derogatory overtones of 'domestic conflict'". Bourke also praised the story's tension, calling it "so well-strung the prose practically vibrates under its influence". A review for Kirkus stated that the novel is a "beautifully realized story and the characters, plot, theme and writing are expertly crafted." === Awards ===
Supertoroid
In geometry and computer graphics, a supertoroid or supertorus is usually understood to be a family of doughnut-like surfaces (technically, a topological torus) whose shape is defined by mathematical formulas similar to those that define the superellipsoids. The plural of "supertorus" is either supertori or supertoruses. The family was described and named by Alan Barr in 1994. Barr's supertoroids have been fairly popular in computer graphics as a convenient model for many objects, such as smooth frames for rectangular things. One quarter of a supertoroid can provide a smooth and seamless 90-degree joint between two superquadric cylinders. However, they are not algebraic surfaces (except in special cases). == Formulas == Alan Barr's supertoroids are defined by parametric equations similar to the trigonometric equations of the torus, except that the sine and cosine terms are raised to arbitrary powers. Namely, the generic point P(u, v) of the surface is given by P ( u , v ) = ( X ( u , v ) Y ( u , v ) Z ( u , v ) ) = ( ( a + C u s ) C v t ( b + C u s ) S v t S u s ) {\displaystyle P(u,v)=\left({\begin{array}{c}X(u,v)\\Y(u,v)\\Z(u,v)\end{array}}\right)=\left({\begin{array}{c}(a+C_{u}^{s})C_{v}^{t}\\(b+C_{u}^{s})S_{v}^{t}\\S_{u}^{s}\end{array}}\right)} where C θ ε = sgn ( cos θ ) | cos θ | ε , S θ ε = sgn ( sin θ ) | sin θ | ε , {\displaystyle {\begin{aligned}C_{\theta }^{\varepsilon }&=\operatorname {sgn} (\cos \theta )\,\left|\,\cos \theta \,\right|^{\varepsilon },\\S_{\theta }^{\varepsilon }&=\operatorname {sgn} (\sin \theta )\ \left|\,\sin \theta \ \right|^{\varepsilon },\end{aligned}}} sgn is the sign function, and the parameters u, v range from 0 to 360 degrees (0 to 2π radians). In these formulas, the parameter s > 0 controls the "squareness" of the vertical sections, t > 0 controls the squareness of the horizontal sections, and a, b ≥ 1 are the major radii in the x and y directions. With s = t = 1 and a = b = R one obtains the ordinary torus with major radius R and minor radius 1, with the center at the origin and rotational symmetry about the z-axis. In general, the supertorus defined as above spans the intervals: − ( a + 1 ) ≤ x ≤ + ( a + 1 ) − ( b + 1 ) ≤ y ≤ + ( b + 1 ) − 1 ≤ z ≤ + 1 {\displaystyle {\begin{array}{rcccl}-(a+1)&\leq &x&\leq &+(a+1)\\[4pt]-(b+1)&\leq &y&\leq &+(b+1)\\[4pt]-1&\leq &z&\leq &+1\end{array}}} The whole shape is symmetric about the planes x = 0, y = 0, and z = 0. The hole runs in the z direction and spans the intervals − ( a − 1 ) ≤ x ≤ + ( a − 1 ) − ( b − 1 ) ≤ y ≤ + ( b − 1 ) − ∞ ≤ z ≤ + ∞ {\displaystyle {\begin{array}{rcccl}-(a-1)&\leq &x&\leq &+(a-1)\\[4pt]-(b-1)&\leq &y&\leq &+(b-1)\\[4pt]-\infty &\leq &z&\leq &+\infty \end{array}}} A curve of constant u on this surface is a horizontal Lamé curve with exponent 2 t , {\displaystyle {\tfrac {2}{t}},} scaled in x and y and displaced in z. A curve of constant v, projected on the plane x = 0 or y = 0, is a Lamé curve with exponent 2 s , {\displaystyle {\tfrac {2}{s}},} scaled and horizontally shifted. If v = 0, the curve is planar and spans the intervals: a − 1 ≤ x ≤ a + 1 − 1 ≤ z ≤ + 1 {\displaystyle {\begin{array}{rcccl}a-1&\leq &x&\leq &a+1\\[4pt]-1&\leq &z&\leq &+1\end{array}}} and similarly if v = 90°, 180°, 270°. The curve is also planar if a = b. In general, if a ≠ b and v is not a multiple of 90 degrees, the curve of constant v will not be planar; and, conversely, a vertical plane section of the supertorus will not be a Lamé curve. The basic supertoroid shape defined above is often modified by non-uniform scaling to yield supertoroids of specific width, length, and vertical thickness. == Plotting code == The following GNU Octave code generates plots of a supertorus:
Model collapse
Model collapse, also known by other names such as "AI inbreeding", "AI cannibalism", "Habsburg AI", and "model autophagy disorder" or "MAD" is a phenomenon noted in artificial intelligence studies, where machine learning models gradually degrade due to errors coming from uncurated synthetic data, or due to training on the outputs of another model such as prior versions of itself. It is unclear to what extent the phenomenon threatens the long-term development of such models, and some techniques have been proposed to mitigate the effect. == Characteristics == Shumailov et al. coined the term to describe two specific stages to the degradation of machine learning models: early model collapse and late model collapse: In early model collapse, the model begins losing information about the tails of the distribution – mostly affecting minority data. Later work highlighted that early model collapse is hard to notice, since overall performance may appear to improve, while the model loses performance on minority data. In late model collapse, the model loses a significant proportion of its performance, confusing concepts and losing most of its variance. == Mechanism == Using synthetic data as training data can lead to issues with the quality and reliability of the trained model. Model collapse occurs for three main reasons: functional approximation errors sampling errors learning errors Importantly, it happens in even the simplest of models, where not all of the error sources are present. In more complex models the errors often compound, leading to faster collapse. == Disagreement over real-world impact == Some researchers and commentators on model collapse warn that the phenomenon could fundamentally threaten future generative AI development: As AI-generated data is shared on the Internet, it will inevitably end up in future training datasets, which are often crawled from the Internet. If training on "slop" (large quantities of unlabeled synthetic data) inevitably leads to model collapse, this could therefore pose a difficult problem. However, recently, other researchers have disagreed with this argument, showing that if synthetic data accumulates alongside human-generated data, model collapse is avoided. The researchers argue that data accumulating over time is a more realistic description of reality than deleting all existing data every year, and that the real-world impact of model collapse may not be as catastrophic as feared. An alternative branch of the literature investigates the use of machine learning detectors and watermarking to identify model generated data and filter it out. == Mathematical models of the phenomenon == === 1D Gaussian model === In 2024, a first attempt has been made at illustrating collapse for the simplest possible model — a single dimensional normal distribution fit using unbiased estimators of mean and variance, computed on samples from the previous generation. To make this more precise, we say that original data follows a normal distribution X 0 ∼ N ( μ , σ 2 ) {\displaystyle X^{0}\sim {\mathcal {N}}(\mu ,\sigma ^{2})} , and we possess M 0 {\displaystyle M_{0}} samples X j 0 {\displaystyle X_{j}^{0}} for j ∈ { 1 , … , M 0 } {\displaystyle j\in {\{\,1,\dots ,M_{0}\,{}\}}} . Denoting a general sample X j i {\displaystyle X_{j}^{i}} as sample j ∈ { 1 , … , M i } {\displaystyle j\in {\{\,1,\dots ,M_{i}\,{}\}}} at generation i {\displaystyle i} , then the next generation model is estimated using the sample mean and variance: μ i + 1 = 1 M i ∑ j X j i ; σ i + 1 2 = 1 M i − 1 ∑ j ( X j i − μ i + 1 ) 2 . {\displaystyle \mu _{i+1}={\frac {1}{M_{i}}}\sum _{j}X_{j}^{i};\quad \sigma _{i+1}^{2}={\frac {1}{M_{i}-1}}\sum _{j}(X_{j}^{i}-\mu _{i+1})^{2}.} Leading to a conditionally normal next generation model X j i + 1 | μ i + 1 , σ i + 1 ∼ N ( μ i + 1 , σ i + 1 2 ) {\displaystyle X_{j}^{i+1}|\mu _{i+1},\;\sigma _{i+1}\sim {\mathcal {N}}(\mu _{i+1},\sigma _{i+1}^{2})} . In theory, this is enough to calculate the full distribution of X j i {\displaystyle X_{j}^{i}} . However, even after the first generation, the full distribution is no longer normal: It follows a variance-gamma distribution. To continue the analysis, instead of writing the probability density function at each generation, it is possible to explicitly construct them in terms of independent random variables using Cochran's theorem. To be precise, μ 1 {\displaystyle \mu _{1}} and σ 1 {\displaystyle \sigma _{1}} are independent, with μ 1 ∼ N ( μ , σ 2 M 0 ) {\displaystyle \mu _{1}\sim {\mathcal {N}}\left(\mu ,{\frac {\sigma ^{2}}{M_{0}}}\right)} and ( M 0 − 1 ) σ 1 2 ∼ σ 2 Γ ( M 0 − 1 2 , 1 2 ) {\displaystyle (M_{0}-1)\,\sigma _{1}^{2}\sim \sigma ^{2}\,\Gamma \left({\frac {M_{0}-1}{2}},{\frac {1}{2}}\right)} , following a Gamma distribution. Denoting with Z {\displaystyle Z} Gaussian random variables distributed according to N ( 0 , 1 ) {\displaystyle {\mathcal {N}}(0,1)} and with S i {\displaystyle S^{i}} random variables distributed with 1 M i − 1 − 1 Γ ( M i − 1 − 1 2 , 1 2 ) {\displaystyle {\frac {1}{M_{i-1}-1}}\Gamma \left({\frac {M_{i-1}-1}{2}},{\frac {1}{2}}\right)} , it turns out to be possible to write samples at each generation as X j 0 = μ + σ Z j 0 , {\textstyle X_{j}^{0}=\mu +\sigma Z_{j}^{0},} X j 1 = μ + σ M 0 Z 1 + σ S 1 Z j 1 , {\textstyle X_{j}^{1}=\mu +{\frac {\sigma }{\sqrt {M_{0}}}}Z^{1}+\sigma {\sqrt {S^{1}}}Z_{j}^{1},} and more generally X j n = μ + σ M 0 Z 1 + σ M 1 S 1 Z 2 + ⋯ + σ M n − 1 S 1 × ⋯ × S n − 1 Z n + σ S 1 × ⋯ × S n Z j n . {\displaystyle X_{j}^{n}=\mu +{\frac {\sigma }{\sqrt {M_{0}}}}Z^{1}+{\frac {\sigma }{\sqrt {M_{1}}}}{\sqrt {S^{1}}}Z^{2}+\dots +{\frac {\sigma }{\sqrt {M_{n-1}}}}{\sqrt {S^{1}\times \dots \times S^{n-1}}}Z^{n}+\sigma {\sqrt {S^{1}\times \dots \times S^{n}}}Z_{j}^{n}.} Note, that these are not joint distributions, as Z n {\displaystyle Z^{n}} and S n {\displaystyle S^{n}} depend directly on Z j n − 1 {\displaystyle Z_{j}^{n-1}} , but when considering X j n {\displaystyle X_{j}^{n}} on its own the formula above provides all the information about the full distribution. To analyse the model collapse, we can first calculate variance and mean of samples at generation n {\displaystyle n} . This would tell us what kind of distributions we expect to arrive at after n {\displaystyle n} generations. It is possible to find its exact value in closed form, but the mean and variance of the square root of gamma distribution are expressed in terms of gamma functions, making the result quite clunky. Following, it is possible to expand all results to second order in each of 1 / M i {\displaystyle 1/M_{i}} , assuming each sample size to be large. It is then possible to show that 1 σ 2 Var ( X j n ) = 1 M 0 + 1 M 1 + ⋯ + 1 M n − 1 + 1 + O ( M i − 2 ) . {\displaystyle {\frac {1}{\sigma ^{2}}}\operatorname {Var} (X_{j}^{n})={\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n-1}}}+1+{\mathcal {O}}\left(M_{i}^{-2}\right).} And if all sample sizes M i = M {\displaystyle M_{i}=M} are constant, this diverges linearly as n → ∞ {\displaystyle n\to \infty } : Var ( X j n ) = σ 2 ( 1 + n M ) ; E ( X j n ) = μ . {\displaystyle \operatorname {Var} (X_{j}^{n})=\sigma ^{2}\left(1+{\frac {n}{M}}\right);\quad \mathbb {E} (X_{j}^{n})=\mu .} This is the same scaling as for a single dimensional Gaussian random walk. However, divergence of the variance of X j n {\displaystyle X_{j}^{n}} does not directly provide any information about the corresponding estimates of μ n + 1 {\displaystyle \mu _{n+1}} and σ n + 1 {\displaystyle \sigma _{n+1}} , particularly how different they are from the original μ {\displaystyle \mu } and σ {\displaystyle \sigma } . It turns out to be possible to calculate the distance between the true distribution and the approximated distribution at step n + 1 {\displaystyle n+1} , using the Wasserstein-2 distance (which is also sometimes referred to as risk): E [ W 2 2 ( N ( μ , σ 2 ) , N ( μ n + 1 , σ n + 1 2 ) ) ] = 3 2 σ 2 ( 1 M 0 + 1 M 1 + ⋯ + 1 M n ) + O ( M i − 2 ) , {\displaystyle \mathbb {E} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {3}{2}}\sigma ^{2}\left({\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n}}}\right)+{\mathcal {O}}\left(M_{i}^{-2}\right),} Var [ W 2 2 ( N ( μ , σ 2 ) , N ( μ n + 1 , σ n + 1 2 ) ) ] = 1 2 σ 4 ( 3 M 0 2 + 3 M 1 2 + ⋯ + 3 M n 2 + ∑ i ≠ j 4 M i M j ) + O ( M i − 3 ) . {\displaystyle \operatorname {Var} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {1}{2}}\sigma ^{4}\left({\frac {3}{M_{0}^{2}}}+{\frac {3}{M_{1}^{2}}}+\dots +{\frac {3}{M_{n}^{2}}}+\sum _{i\neq j}{\frac {4}{M_{i}M_{j}}}\right)+{\mathcal {O}}\left(M_{i}^{-3}\right).} This directly shows why model collapse occurs in this simple model. Due to errors from re-sampling the approximated distribution, each generation ends up corresponding to a