:[gT,dZddlZddlZddlZddlmZmZmZmZm Z m Z m Z m Z m Z ddlZddlmZmZmZddlmZmZmZmZddlmZmZmZddlmZmZm Z m!Z!m"Z"m#Z#e d Z$d e e d e e d e e fd Z%e# d0ddddddddddd dee&efdede'de e e ee&fde e!d e e de e(de e'de ejRde e e(e'fde e e&ejTee+fde e ed e e d efdZ,Gdd Z-GddZ.deej^d ej^d ej^fd!Z0 d1d"ed#e'd$ede e&d%e ed&e(d ee-fd'Z1 d2d"ed#e'd$ed(e'de e&d%e ed)e e(d*e ed&e(d ee-fd+Z2 d3dedede'd#e'd)e(d*ed,e e&de e!d e e de e(de e'd%e ed-e(de e e'e(fd.e(d(e'de e ed&e(d e e d e ee&e3feff(d/Z4y)4z.Training Library containing training routines.N) AnyDictIterableListOptionalSequenceTupleUnioncast) BoosterParamCallableFPreProcCallable)CallbackContainer EarlyStoppingEvaluationMonitorTrainingCallback)SKLEARN_INSTALLED DataFrameXGBStratifiedKFold)BoosterDMatrixMetric Objective XGBoostError_deprecate_positional_argsCVPackfeval custom_metricreturncn|d}tjd|d| | td||}|S|}|S)NzIhttps://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.htmlzy`feval` is deprecated, use `custom_metric` instead. They have different behavior when custom objective is also used.See z$ for details on the `custom_metric`.zLBoth `feval` and `custom_metric` are supplied. Use `custom_metric` instead.)warningswarn ValueError)rrlink eval_metrics K/var/www/html/bid-api/venv/lib/python3.12/site-packages/xgboost/training.py_configure_custom_metricr(sp  W    &< >  ]6 Z  $1#<-K CHK T) evalsobjrmaximizeearly_stopping_rounds evals_result verbose_eval xgb_model callbacksrparamsdtrainnum_boost_roundr*r+r,r-r.r/r0r1c | gntjt| } t|| } |r t|ng}t||g|Dcgc]}|d c}z| }d}| r#| durdn| } | j t | |r| j t ||t| | t|xs| |u}|j|}t||D]B}|j||||rn,|j||||j||||sBn|j|}||j|j|jScc}w) a Train a booster with given parameters. Parameters ---------- params : Booster params. dtrain : Data to be trained. num_boost_round : Number of boosting iterations. evals : List of validation sets for which metrics will evaluated during training. Validation metrics will help us track the performance of the model. obj Custom objective function. See :doc:`Custom Objective ` for details. feval : .. deprecated:: 1.6.0 Use `custom_metric` instead. maximize : Whether to maximize feval. early_stopping_rounds : Activates early stopping. Validation metric needs to improve at least once in every **early_stopping_rounds** round(s) to continue training. Requires at least one item in **evals**. The method returns the model from the last iteration (not the best one). Use custom callback or model slicing if the best model is desired. If there's more than one item in **evals**, the last entry will be used for early stopping. If there's more than one metric in the **eval_metric** parameter given in **params**, the last metric will be used for early stopping. If early stopping occurs, the model will have two additional fields: ``bst.best_score``, ``bst.best_iteration``. evals_result : This dictionary stores the evaluation results of all the items in watchlist. Example: with a watchlist containing ``[(dtest,'eval'), (dtrain,'train')]`` and a parameter containing ``('eval_metric': 'logloss')``, the **evals_result** returns .. code-block:: python {'train': {'logloss': ['0.48253', '0.35953']}, 'eval': {'logloss': ['0.480385', '0.357756']}} verbose_eval : Requires at least one item in **evals**. If **verbose_eval** is True then the evaluation metric on the validation set is printed at each boosting stage. If **verbose_eval** is an integer then the evaluation metric on the validation set is printed at every given **verbose_eval** boosting stage. The last boosting stage / the boosting stage found by using **early_stopping_rounds** is also printed. Example: with ``verbose_eval=4`` and at least one item in **evals**, an evaluation metric is printed every 4 boosting stages, instead of every boosting stage. xgb_model : Xgb model to be loaded before training (allows training continuation). callbacks : List of callback functions that are applied at end of each iteration. It is possible to use predefined callbacks by using :ref:`Callback API `. .. note:: States in callback are not preserved during training, which means callback objects can not be reused for multiple training sessions without reinitialization or deepcopy. .. code-block:: python for params in parameters_grid: # be sure to (re)initialize the callbacks before each run callbacks = [xgb.callback.LearningRateScheduler(custom_rates)] xgboost.train(params, Xy, callbacks=callbacks) custom_metric: .. versionadded 1.6.0 Custom metric function. See :doc:`Custom Metric ` for details. Returns ------- Booster : a trained booster model r) model_fileTr )periodroundsr,)metric output_margin) iterationfobj)copylistr(rappendrrrcallablebefore_trainingrangebefore_iterationupdateafter_iterationafter_traininghistory)r2r3r4r*r+rr,r-r.r/r0r1r metric_fndbststart_iteration cb_containeris r'trainrO3shP 'TYYtI-GI( >I DKbE &6(E%:EqadE%::y QCO(D0ql *,?@.ChWX$sm9yE'9 L  & &s +C ?O 4  ( (a ?  6QS 1  ' 'Q >  5  % %c *CL001 88:C&;s E# c eZdZdZdededeeeefddfdZ de de fd Z d e d eeddfd Zd e d eedede fdZy)rz. "Auxiliary datastruct to hold one fold of CV.r3dtestparamr Nc`||_||_|df|dfg|_t|||g|_y)z "Initialize the CVPackrOtestN)r3rQ watchlistrrK)selfr3rQrRs r'__init__zCVPack.__init__s9  !7+eV_=565/2r)namec<dtdtdtffd }|S)Nargskwargsr c<tj|i|SN)getattrrK)rZr[rXrVs r'_innerz"CVPack.__getattr__.._inners *7488T*D;F; ;r))r)rVrXr_s`` r' __getattr__zCVPack.__getattr__s$ <# < < < r)r<r=cR|jj|j||y)z' "Update the boosters for one iterationN)rKrEr3)rVr<r=s r'rEz CVPack.updates  Y5r)rr;cR|jj|j|||S)z( "Evaluate the CVPack for one iteration.)rKeval_setrU)rVr<rr;s r'evalz CVPack.evals!xx  E=QQr))__name__ __module__ __qualname____doc__rrr rrrWstrrr`intrrErboolrdr)r'rrs833&-36>uT4Z?P6Q3 3 668I+>646RcR(6*:R4RTWRr)c TeZdZdeddfdZdedeeddfdZdedee d e de e fd Z d eedefd Zd e dee fdZ ddeeeee efe fdee ddfdZdefdZedefdZej2deddfdZedefdZej2deddfdZy)_PackedBoostercvfoldsr Nc||_yr])ro)rVros r'rWz_PackedBooster.__init__s  r)r<r+cJ|jD]}|j||y)z Iterate through folds for updateN)rorE)rVr<r+folds r'rEz_PackedBooster.updatesLLD KK 3 '!r)rr;cd|jDcgc]}|j|||}}|Scc}w)zIterate through folds for eval)rord)rVr<rr;fresults r'rdz_PackedBooster.evals6DH<<P>PVWPV1299Z]Jq1u4EFPVW XXWs&AdallnfoldrRfpreprocshufflec |jd}tj|}|dur)tjj t |}ntj t |}tj||} t|D cgc]7} tjt|D cgc] } | | k7s | | c} 9} } } | D cgc]} t| |}} | Dcgc]}t||}}g}t|D]} |j|| d}|j|| | |j|| d}|j|| | |||||j\}}}n|}t|j|Dcgc]}d|fc}z}|j!t#||||Scc} wcc} } wcc} wcc}wcc}w)zb Make n folds for cross-validation maintaining groups :return: cross-validation folds group_ptrT) allow_groupsr&) get_uint_infordiffrandom permutationlenr array_splitrCrrslice set_groupr>r?itemsr@r)rrrRr*rrgroup_boundaries group_sizesidxout_group_idsetkrNin_group_idset in_groupsin_idset out_groups out_idsetretr3rQtparamitmplsts r' mkgroupfoldr"s))+6''*+K$ii##C $45iiK()nnS%0OuA E%LKLqAF*LKL FTES y"23^ HWGVz#34 C 5\HQKd;^A%678 9Qrr@r)rrrRrr*rrrrrrrrNrxsplitssfkrr3rQrrrs r'mknfoldrWs KEIINN4Uu} t!!+. /! 3eU%(G  d?))'' 7C))DLLN+CNN3. 5\ ! NN%,I,Q!q&IaL,I J!     /&+,e!eH,',-u!1uI- I!%DQcii$.."2dnn6FiGH"()&QAaD&)#)*6aQqT6 *I C 5\HQK( 9Q<(  $,VUEJJL$I !FE6FFLLN#u&Mu s';u&MM 6&%./ JEJ -- /%++(8DNN$rY#wxYw) a Cross-validation with given parameters. Parameters ---------- params : dict Booster params. dtrain : DMatrix Data to be trained. num_boost_round : int Number of boosting iterations. nfold : int Number of folds in CV. stratified : bool Perform stratified sampling. folds : a KFold or StratifiedKFold instance or list of fold indices Sklearn KFolds or StratifiedKFolds object. Alternatively may explicitly pass sample indices for each fold. For ``n`` folds, **folds** should be a length ``n`` list of tuples. Each tuple is ``(in,out)`` where ``in`` is a list of indices to be used as the training samples for the ``n`` th fold and ``out`` is a list of indices to be used as the testing samples for the ``n`` th fold. metrics : string or list of strings Evaluation metrics to be watched in CV. obj : Custom objective function. See :doc:`Custom Objective ` for details. feval : function .. deprecated:: 1.6.0 Use `custom_metric` instead. maximize : bool Whether to maximize feval. early_stopping_rounds: int Activates early stopping. Cross-Validation metric (average of validation metric computed over CV folds) needs to improve at least once in every **early_stopping_rounds** round(s) to continue training. The last entry in the evaluation history will represent the best iteration. If there's more than one metric in the **eval_metric** parameter given in **params**, the last metric will be used for early stopping. fpreproc : function Preprocessing function that takes (dtrain, dtest, param) and returns transformed versions of those. as_pandas : bool, default True Return pd.DataFrame when pandas is installed. If False or pandas is not installed, return np.ndarray verbose_eval : bool, int, or None, default None Whether to display the progress. If None, progress will be displayed when np.ndarray is returned. If True, progress will be displayed at boosting stage. If an integer is given, progress will be displayed at every given `verbose_eval` boosting stage. show_stdv : bool, default True Whether to display the standard deviation in progress. Results are not affected, and always contains std. seed : int Seed used to generate the folds (passed to numpy.random.seed). callbacks : List of callback functions that are applied at end of each iteration. It is possible to use predefined callbacks by using :ref:`Callback API `. .. note:: States in callback are not preserved during training, which means callback objects can not be reused for multiple training sessions without reinitialization or deepcopy. .. code-block:: python for params in parameters_grid: # be sure to (re)initialize the callbacks before each run callbacks = [xgb.callback.LearningRateScheduler(custom_rates)] xgboost.train(params, Xy, callbacks=callbacks) shuffle : bool Shuffle data before creating folds. custom_metric : .. versionadded 1.6.0 Custom metric function. See :doc:`Custom Metric ` for details. Returns ------- evaluation history : list(string) Tz;sklearn needs to be installed in order to use stratified cvrr&r N)r7rr8)r:is_cvr;z-meanz-std)!rr isinstancerir>r?dictpoprr(r@rrrrArnrBrCrDrErF aggregated_cvr rr rkeysrpandasr from_dict ImportErrorrG)"r2r3r4rrrrr+rr,r-rrr/rrr1rrr_metricsresultsrorIcallbacks_containerboosterrN should_breakresrxmeanstdrpds" r'cvrsZT"3 I  '3) [[]F&$"(B&QAaDM,AAaD&Bf F "$,F= ! &0 f]+T 2]+Gm,-G JJ}d#&(GvtWh E7G) >I 'TYYtI-GI(D0ql *,)TU.ChWX+sm9yE'9 W%G''0 ? #  / /FD I q#*::7AvtT !//"4c5%.?(@#A3GNCsW}G+)+g &V|7*(*f % C'M " ) )$ / C&L ! ( ( - H \\^$QZ(F7+A+AA+EG $ %$&  ll,,W5G&&w/ NGC|   s J,J,:J11 J=<J=) )rlNT)rlNFNT)rFNrlNNNNNTNTrNTN)5rhr>osr"typingrrrrrrr r r numpyr_typingr rrcallbackrrrrcompatrrrcorerrrrrrrr(rirjrkEvalsLogPathLike bytearrayrOrrnndarrayrrrrrrlr)r'rsa5 TTT== ED H  F ,4V,< f(L 6:#"#+/8