As zero-shot performance gains flatten, researchers are rethinking the field: separating architectural progress from data-driven gains, and introducing a language-driven paradigm for TSFMs.
Ask a power grid operator, a hospital administrator, or a logistics planner what keeps them up at night, and somewhere on the list will be a forecasting problem. How much electricity will customers draw next Tuesday? How many patients will arrive in the emergency department this weekend? How many containers will move through the port next month? For decades, these forecasts have been built using custom statistical models for each application. Each requires its own data, its own tuning, its own expert.
Over the past few years, the machine learning community has been chasing a more ambitious idea: what if a single model, trained once on a vast and diverse collection of historical time series, could forecast anything, including data from domains that it has never seen during training? This is the premise of a time-series foundation model (TSFM). TSFM borrows ideas from large language models: train the models on enough data of the right kind and the models will pick up general patterns, allowing them to produce accurate forecasts in new domains without retraining.
The challenge
The standard test of this capability is called zero-shot forecasting: the model is shown a stretch of history from a dataset it has never encountered during training and asked to predict what comes next without fine-tuning or domain-specific adjustment. Performance is scored against ground truth future values using some standardized metrics, such as mean absolute scaled error (MASE) for point predictions and continuous ranked probability score (CRPS) for the model’s full forecast distribution. The GIFT-Eval benchmark, which aggregates 97 test cases across seven domains, has become the field’s de facto leaderboard.
That leaderboard has filled up quickly with models such as Chronos and Chronos-2 from Amazon, TimesFM from Google, Moirai from Salesforce, and others. Each new release claims some architectural innovation that leads to stronger performance. The pace of progress is real. But it has produced a maturing field with two problems that are harder to see while the leaderboard is still moving.
When pretraining corpus, training protocol, and architecture all change together, it becomes impossible to say which one is mostly responsible for the performance gain.
The first problem is identifying the main contributors to the performance gain. When one model outperforms another on GIFT-Eval, the natural assumption is that the new architecture is responsible. Yet each team trains its model on a different mix of data, with different recipes, often using proprietary synthetic data generation pipelines that are described only in broad strokes. When pretraining corpus, training protocol, and architecture all change together, it becomes impossible to say which one is mostly responsible for the performance gain. The community has been comparing combined systems but interpreting the comparisons as if they isolated architectural progress.
The second is a capability problem, and it becomes more pressing as accuracy on the leaderboard saturates. The standard TSFM interface is narrow: a window of history goes in, a forecast distribution comes out. There is no clean channel through which a human, or another automated system, can guide the prediction using external knowledge or contextual reasoning. As leaderboard differences shrink to fractions of a percent, the more useful question is shifting from how to forecast slightly more accurately to accomplishing what these models cannot yet do at all.
Time-Series Data Agent
The work described in this article emerged from a project at Rensselaer Polytechnic Institute, conducted jointly with collaborators at IBM Research and supported by the National AI Research Resource (NAIRR) Pilot and the Mass Open Cloud. The broader project aims to develop an agentic system, the Time-Series Data Agent, that couples a time-series foundation model with a large language model to enable language-driven forecasting, analysis, and control. Building that system required progress on both problems above. Before committing the project to a particular foundation-model backbone, the team needed to know which TSFM design choices actually matter and which are incidental. And before that backbone could be coupled meaningfully to a language model, the team needed a representation through which the two systems could interact. Two parallel lines of work addressed these questions. The first asked how much of recent TSFM progress is really attributable to architecture. The second asked what new capabilities become possible if the representation of time series inside the model is itself reconsidered. The findings from each, taken together, map a position for the field and a path for the larger project.
Architecture vs. data
To address the first question, the team built a deliberately unremarkable forecasting model around a standard encoder-only Transformer backbone, based on the PatchTST architecture developed earlier by the IBM collaborators. There were no time-series-specific modifications to the attention mechanism and no novel components in the backbone itself. The surrounding pieces—patch-based tokenization of the input sequence, a normalization scheme that handles missing values, and a quantile-based output head for probabilistic predictions—were all assembled from techniques already established in the literature. The model was paired with a carefully documented training recipe built entirely from publicly available data, then compared against the specialized models at the top of the leaderboard.
The team’s generic transformer-based model, PatchTST-FM, delivered top performance. It matched or exceeded the results of leading specialized TSFMs on both the GIFT-Eval and TIME benchmarks. Notably, among the top zero-shot performers on the GIFT-Eval leaderboard—excluding those with training data leakage—PatchTST-FM outperformed models like FlowState (IBM Research), TimesFM-2.5 (Google Research), TiRex (NXAI), and Chronos-2-Synth (Amazon) by a small margin.
Many of the architectural innovations claimed over the past two years are likely carrying credit that belongs to differences in data and training.
Three factors contributed to the performance gain: the pretraining data, the masking strategy during pretraining, and the context-window length. The cumulative message is that a generic Transformer trained well on the right data, with contiguous masking and a long context window, is sufficient to compete with, and even beat, TSFMs that introduce specialized architectural components. The corollary is uncomfortable: many of the architectural innovations claimed over the past two years are likely carrying credit that belongs to differences in data and training. The team released their model checkpoints and full training pipeline as a public baseline, in part to make this kind of confounding harder to introduce in future work.
Single- vs. multi-scale
The findings above sharpened the second question. If the frontier of high-performance models is becoming densely packed, the productive question is no longer how to squeeze out a small gain of performance, but how to engineer new functionalities. In particular, the typical TSFMs currently offer no clean and practical method of intervention. A continuous patch architecture exposes only raw numerical values, which is the wrong level of abstraction for either human or machine reasoning to operate at. To couple a TSFM with a language model in any meaningful way, the representation itself needs to change.
The second line of work, an architecture named ASTEP, for Abstract Semantic Tokenizer-Predictor, can facilitate intervention. The central idea is to translate the series into a sequence of discrete tokens drawn from a learned vocabulary of recurring temporal “shapes” and to use that vocabulary to describe time-series data across multiple scales. Coarse tokens describe broad structures, such as a gradual rise or a long plateau. Finer tokens describe the smaller fluctuations layered on top. The series is encoded as a coarse-to-fine hierarchy, in a manner loosely analogous to how text is encoded as sentences, words, and sub-word pieces. What is interesting about ASTEP is not discrete tokenization itself, which has been explored before, but the multi-scale residual structure.
Ablations show that a single-scale tokenizer operating only at patch resolution performs measurably worse than the multi-scale version. The hierarchy is doing real work. The plausible reason is that time series carry meaningful structure at many resolutions at once; for example, a daily oscillation rides on a weekly cycle, which rides on a seasonal trend, which rides on a longer-term trend, and so on. A flat tokenization forces the model to recover this hierarchy implicitly. A multi-scale tokenization makes it explicit in the representation, which has the further effect of making the model’s internal reasoning legible in a way continuous-patch models are not: a coarse token corresponds to an interpretable structural decision, not an opaque vector.
Results
The accuracy result is competitive. Even without intervention, ASTEP’s large variant model achieves the best performance in zero-shot GIFT-Eval benchmarking, beating PatchTST-FM. Further, because the model represents forecasting as a hierarchy of tokens, a user or an external system can specify the coarse-level tokens directly and let the model fill in the fine detail consistent with that sketch. The team calls this concept-level intervention. Empirically, it produces little benefit at short horizons, where there is no coarse structure left to specify. However, at long horizons where unconstrained forecasts diverge and uncertainty grows, high-level intervention translates into substantial gains in accuracy.
High-level intervention translates into substantial gains in accuracy.
This is the piece that completes the picture for the larger project. A language model that has understood a user’s request, or that has reasoned through a planning problem, has no clean mechanism for pushing its conclusions into a conventional TSFM beyond rewriting the input or post-editing the output. ASTEP’s token hierarchy provides one. The language model can specify the coarse tokens, e.g., “expect a slow rise, then a plateau.” ASTEP can produce the calibrated, fine-grained probabilistic forecast consistent with that intervention.
Taken together, the two lines of work map a position for the field. The first sets the floor: with a careful, public training recipe, even a generic architecture is competitive with the best-performing specialized models on leaderboards, and many recent leaderboard movements may not reflect genuine architectural progress. The second raises the ceiling: discrete multi-scale tokenization delivers comparable accuracy while adding a new axis of capability, controllable forecasting, that standard designs cannot support. For the sponsors of the project, the value is on both fronts. A transparent baseline lowers the cost of deploying a strong forecasting system in any domain. A foundation model that can be steered by language opens new patterns of use in planning, monitoring, and decision support, which is the eventual objective of Time-Series Data Agents.