Improving tie breaking in Development() drop_high/low

[kennethshsu]

Description

Originally posted by @kennethshsu in #1009 (comment)

The drop_high/drop_low can be improved.

Is your feature request aligned with the scope of the package?

• Yes, absolutely!
• No, but it's still worth discussing.
• N/A (this request is not a codebase enhancement).

Describe the solution you'd like, or your current workaround.

Maybe a way to customize which one is actually dropped/included? I personally think the factors with the least amount of "data" or losses should be dropped.

Check this test.

Do you have any additional supporting notes?

No response

Would you be willing to contribute this ticket?

• Yes, absolutely!
• Yes, but I would like some help.
• No.

[henrydingliu]

for starters, i would argue that the current behavior of Development is what you would want in practice. we SHOULD be dropping the outlier with the most weight, for whatever reason that's making us want to drop outliers in the first place.

however, more granular control over dropping is possible through the new TriangleWeight class.

print(cl.TriangleWeight(drop_high=1).fit(
    tri.age_to_age,
    sample_weight=tri[tri.valuation.year < 1985] * -1
).w_)

      12-24  24-36  36-48  48-60
1981    NaN    NaN    NaN    NaN
1982    1.0    NaN    NaN    NaN
1983    1.0    NaN    NaN    NaN
1984    1.0    NaN    NaN    NaN

When we eventually migrate the internals of DevelopmentBase to use TriangleWeight, we can add a new parameter to Development to drop the earliest/latest/largest/smallest as the tiebreaker.

of course, we could always add that parameter right now and just make changes to the existing DevelopmentBase logic.