ENH: Make serious FG-BG-WTHH creation exogenous

[MImmesberger]

Is your feature request related to a problem?

We cannot deal with groups that depend on Vorrang-/Günstigerprüfungen across group constellations with our static DAG. Rather than attempting to create group creation functions which call GETTSIM recursively, we should provide a repository with a recipe to call GETTSIM repeatedly.

Based on the discussion below, the most promising strategy is:

1. The functions fg_id, bg_id, and wthh_id all return the same thing.
2. If they are called, GETTSIM issues a warning to users that the results will likely be wrong in some common scenarios, pointing to the external repo. Things like eigenbedarf_gedeckt will disappear.
3. The external repo contains the code that we have now plus, eventually, the work from Endogenous Bedarfsgemeinschaften #778. Calling the main function in that repo will produce "correct" fg_id, bg_id, wthh_id and results of Vorrang-/Günstigerprüfungen (e.g., wohngeld_vorrangig_oder_günstiger), which can be used as part of the input_data in a subsequent GETTSIM call.
4. All other functions in GETTSIM remain the same, in particular they will assume that fg_id, bg_id, and wthh_id are correct no matter where they come from.

HMG: edited so that it reflects my understanding of the outcomes of our discussion below, please double-check @MImmesberger

[MImmesberger]

We can leave the bg_id and wthh_id functions in GETTSIM. What we will not determine endogenously is whether person X receives SGB II or WoGG transfers. Rather, we will have input variables beantragt_wohngeld_wthh and beantragt_arbeitslosengeld_2_bg. If both are false for individual X, it does not get any transfer. Max one of them can be true for one individual. These two input booleans replace much of what is currently called eigenbedarf_gedeckt.

In the other repo, we will determine the two input variables endogenously. For now, we will use the exact logic that is currently implemented in GETTSIM. Later, we can implement what I started in #778

[hmgaudecker]

Can you clarify a little more? That is, beantragt_arbeitslosengeld_2_bg might be infeasible (=ineligible) whereas beantragt_wohngeld_2_bg is possible. So this seems to conflate the administrative rule of eigenbedarf_gedeckt with a behavioral rule. But maybe this is not what you have in mind at all and the name has put me on the wrong track?

My other source of confusion is that you write about the two variables as individual-level, but define them on a level that still needs to be created?

[MImmesberger]

So this seems to conflate the administrative rule of eigenbedarf_gedeckt with a behavioral rule.

Yes, that's true. The idea is that in reality, people have to apply themselves for the correct programme (or not at all). It can still be the case that households then get rejected. It's a good policy input because this is exactly the decision a household takes; optimal decisions (in terms of transfer size) are provided via the additional repo.

My other source of confusion is that you write about the two variables as individual-level

Yes, I wasn't careful there, just took it out of the #778 DAG. Should be individual level.

[hmgaudecker]

It's a good policy input because this is exactly the decision a household takes; optimal decisions (in terms of transfer size) are provided via the additional repo.

I am not sold on this behavior just yet. It will certainly confuse many people who are used to a rule "just take the best thing I can get". Maybe that won't be perfect across possible bg/wthh configurations, but for 99.5% of all cases it is perfectly viable. Not sure we want to throw that possibility out for a corner case.

[MImmesberger]

I think we have three options. Your comment sounds like you prefer (1).

1. Do automatically what is correct for most HHs. This implies we keep the entire endogenous machinery (including the Vorrangprüfungen). Maybe we perform them on hh instead of bg level to make clear that this works only for certain kinds of households.
2. We add a successor for eigenbedarf_gedeckt. If you don't like the approach laid out above, we could add something like wohngeld_vorrangig_oder_günstiger (naming could be improved), where False implies that the individual should receive ALG2.
3. We remove wthh_id and bg_id as group_creation_functions completely. I feel like this is much more complex than 1 or 2, at least for casual users.

[hmgaudecker]

Thanks!

I am not sure I quite understand 1.

My understanding of 2. was that it is not feasible to define that at the individual level because of the nonlinearities in Wohngeld. So it factually ends up being 3.

I still think 3. is the way to go. People outside of family econ / IAB will never notice because in almost all cases, FG = BG = WTHH. One thing we could do, though, is to implement that rule (i.e., FG = BG = WTHH) and issue a warning in case the function is being run. The warning would point to the repo that will seriously set BG and WTHH.

[MImmesberger]

(1) is just status quo! It works well as long as bg=wthh=hh

(2) is feasible. The algorithm that sets the input optimally accounts for this nonlinearity. I think its really the same as (3), just one step "earlier". The only difference is that in (2) users specify bools and we build the IDs, and in (3) users specify the IDs themselves.

in almost all cases, FG = BG = WTHH

You mean we set default IDs and just assume HH = BG = WTHH if no input is provided?

[hmgaudecker]

The status quo still has stuff like eigenbedarf_gedeckt, which is not well-defined.

Having bools instead of ids would just add increase (my, at least) confusion.

in almost all cases, FG = BG = WTHH

You mean we set default IDs and just assume HH = BG = WTHH if no input is provided?

Not quite, we would keep the FG logic and split up multi-gen households (in case HH instead of FG wasn't just a typo).

We would then have functions like:

@group_creation_function(reorder=False)
def bg_id(fg_id: IntColumn) -> IntColumn:
    return fg_id

Could also be a policy_function, but maybe it is clearer that way.

Then have something like this in warn_if:

@warn_function()
def bg_wthh_creation_is_oversimplified(
    specialized_environment__with_partialled_params_and_scalars: SpecEnvWithPartialledParamsAndScalars,
) -> None:
    """Warn if we are using the internal, oversimplified version of group creations."""
    env = specialized_environment__with_partialled_params_and_scalars
    if (
        isinstance(env.get("bg_id"), GroupCreationFunction))
        or isinstance(env.get("wthh_id"), GroupCreationFunction))
    ):
        msg = format_errors_and_warnings(
            """
            You are (likely) using GETTSIM's oversimplified version for constructing 
            `bg_id` (Bedarfsgemeinschaften) and `wthh_id` (wohngeldrechtliche Teilhaushalte).

            This will be wrong for some household configurations. The prime example occurs
            when children have some income from employment or receive alimony payments.

            See https://github.com/ttsim-dev/gettsim-crazy-grouping-rules for a recipe
            implementing a close approximation of the law.
            """
        warnings.warn(
            FunctionsAndDataColumnsOverlapWarning(overridden_elements),
            stacklevel=2,
        )

Btw, I am serious about the repo name (or does it sound too unprofessional? Certainly is fitting...)

[MImmesberger]

The issue I see with this is that we still need to determine which group gets which transfer, given IDs. Let's say we have two FGs and need to decide about which transfers to give them:

• Your proposed algorithm would put them in different BGs (correct), but also different WTHH (correct only if both receive Wohngeld or both receive ALG2).
• More importantly, we're back having the old problem. Even if we did Vorrang/Günstigerprüfungen to determine which transfer to take up, we might be wrong (for both FGs independently, the algorithm might say both Wohngeld would be better, but jointly, it could be worse). We'd basically need to infer how the set of IDs came about, assuming they are correct/optimal.

This is why I favored the wohngeld_vorrangig_oder_günstiger input instead of the IDs. Basically, we'll always need some version of wohngeld_vorrangig_oder_günstiger and I'd rather keep it exogenous than endogenous. The upside is that we can identify the IDs using that.

[hmgaudecker]

This sounds like we need to extend the warning to that case (the problem here being that wthh is not nested within fg, IIUC?).

[hmgaudecker]

Basically, we'll always need some version of wohngeld_vorrangig_oder_günstiger and I'd rather keep it exogenous than endogenous.

Easy to accommodate the best of both worlds: Just spit out that variable from https://github.com/ttsim-dev/gettsim-crazy-grouping-rules, too! Will be exogenous in the correct case, but endogenous for almost everyone.