update
Jun 1, 2026
By Teun
Json2vec turns nested JSON records into model architectures
Json2vec is an open-source PyTorch and Lightning toolkit that builds models directly from JSON-like schemas instead of flattening records first. It supports nested data, typed fields, masked reconstruction, embeddings, and batch inference through Lightning data modules and writers.
Json2vec is an open-source library that builds PyTorch and Lightning models directly from JSON-like schemas, according to the project’s announcement on Hacker News. The tool is aimed at predictive modeling on nested records such as customer histories, orders with line items, sessions, and telemetry data, where a flat table can lose useful structure.
The project takes a different approach from standard machine-learning pipelines that flatten structured records into one row of features. With json2vec, the schema itself becomes both the data contract and the model blueprint, so the shape of the record defines how the model is assembled.
The library maps leaf fields to typed tensor fields. The supported field types listed in the announcement include Number, Category, Set, Entity, Text, and Vector, while Array nodes act as local context encoders for repeated child objects.
Targets, masking, pruning, and embeddings are all configured on the same schema tree. That means a single schema can describe supervised prediction, masked reconstruction, unsupervised embedding generation, field importance, schema mutation, batch inference, and serving.
The project shows this with an order example. In that schema, customer_tier is a category field, line_items is an array of repeated item records, and returned is marked as a target so its value is hidden from the input and decoded as a supervised output.
Prediction results stay tied to schema addresses, according to the project. That lets decoded values and embeddings remain attached to the part of the record that produced them, instead of being detached into an anonymous feature vector.
Json2vec’s author says the library is built around Lightning. j2v.Model is a LightningModule, and the project includes j2v.PolarsDataModule and j2v.StreamingDataModule implementations for in-memory data, local files, and S3-backed datasets.
For small jobs, users can call model.predict on raw Python dictionaries. For offline batch prediction, the project says you can attach j2v.Writer to Lightning’s prediction loop, which writes rank-partitioned Parquet files such as predictions/rank-0.parquet.
The announcement also says json2vec does not split supervised and self-supervised learning into separate code paths. Instead, target=True, p_mask, and p_prune all use the same reconstruction path, with target=True described as shorthand for p_prune=1.0.
The project supports schema evolution after construction, and it distinguishes between null, padded, masked, and valued tensorfield states for missing data handling. It also says the same configured path is used for queries, preprocessing, tensorization, model execution, prediction writing, and postprocessors.
For larger runs, the library works with normal Lightning features such as callbacks, checkpointing, precision settings, device placement, and distributed strategies. The package requires Python 3.12 or newer, and optional extras add text, serving, and documentation dependencies.
Json2vec is open source under the Apache License 2.0. The project says it does not try to replace feature stores, governance systems, rule engines, authorization layers, decision-capture systems, or audit platforms; it focuses on the reusable encoder and runtime layer for nested business data.
