Andreas_Kokolantonakis
06/27/2023, 8:06 AMAndreas_Kokolantonakis
06/27/2023, 8:09 AMJuan Luis
06/27/2023, 8:31 AMso that the catalog paths can point to the specific datedo you mean, you have several catalogs/catalog directories with a date in the filename, and you want to pick the appropriate one from the
kedro run
CLI?Andreas_Kokolantonakis
06/27/2023, 8:41 AMAndreas_Kokolantonakis
06/27/2023, 8:42 AMJuan Luis
06/27/2023, 8:50 AMJuan Luis
06/27/2023, 8:55 AMAndreas_Kokolantonakis
06/27/2023, 8:55 AMJuan Luis
06/27/2023, 8:57 AMOmegaConfigLoader
loadingJuan Luis
06/27/2023, 9:02 AMcars:
type: pandas.CSVDataSet
filepath: data/01_primary/${today:%Y_%m_%d}/cars.csv
and then in `settings.py`:
import datetime as dt
from omegaconf import OmegaConf
def get_today(date_fmt: str) -> str:
return dt.date.today().strftime(date_fmt)
if not OmegaConf.has_resolver("today"):
OmegaConf.register_new_resolver("today", get_date)
(I didn't test this, but you get the idea)
how does that sound?Andreas_Kokolantonakis
06/27/2023, 9:06 AMJuan Luis
06/27/2023, 9:17 AMoc.env
resolver. it's disabled by default but you can enable it.Andreas_Kokolantonakis
06/27/2023, 9:40 AMJuan Luis
06/27/2023, 9:53 AMOmegaConfigLoader
in this case yes