:[g1dZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z m Z ddl m Z ddlmZmZddlmZddlmZmZddlmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$ddl%Z&ddl'Z(d d l)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAd d lBmCZCGd deDZEe$deFdeGfdZHe$deeFdejfdZHde"eFeeFfde"eGejffdZHde0deeeNd?fd/e e&jd@e[deeFe"eNeldfff dAZmd>e/dBes*r9r3datareturncyNr8r;s r:from_pystr_to_cstrr@Bs,/r9cyr>r8r?s r:r@r@Fs9ttj|S)z*Make JSON-based arguments for C functions.)r@jsondumps)rZs r: make_jcargsr^qs djj0 11r9resultc|jdd}|Dcgc]}t|jd}}|Dcgc]\}}|t|f}}}|Scc}wcc}}w)z-Parse an eval result string from the booster.rN:)splittuplefloat)r_splitedsmetric_score_strn metric_scores r:_parse_eval_strrjvsfllnQR G5<=Waggcl+W=.>?.>daQaM.>L? >?s !AA! IterRange expectationsgotcd}tt|dz D]}|t||z }|dz }|t|dz }|dt|zz }|S)zTranslate input error into string. Parameters ---------- expectations : a list of expected value. got : actual input Returns ------- msg: str z Expecting rz or z. Got )rRrJrE)rlrmmsgts r:_expectrrsl C 3|$q( ) s<?## v *3|B  C9s3x C Jr9rpct|}|jddk7rtj|tyt |y)z5Redirect logs from native library into Python consolezWARNING:roN)r0findwarningswarn UserWarningprint)rpsmsgs r: _log_callbackrzs4 #;D yy" dK( $Kr9cbtjdtj}|tS)z2Wrap log_callback() method in ctypes callback typeN)rH CFUNCTYPErIrz) c_callbacks r:_get_log_callback_funcr~s#!!$8J m $$r9libcXtj}tj}tj}|jtj|tj|tj||j|j|jfS)z2Get the XGBoost version from native shared object.)rHc_intXGBoostVersionbyrefrS)rmajorminorpatchs r: _lib_versionrsi LLNE LLNE LLNEv||E*FLL,?eATU ;; U[[ 00r9ctjjtjjtd}t |d5}|j jcdddS#1swYyxYw)z1Get the XGBoost version from Python version file.VERSIONrQencodingN)ospathjoindirname__file__openreadstrip) VERSION_FILEfs r: _py_versionrsK77<< 99EL lW -vvx~~ . - -s A77Bctj|j_t |_|j |j dk7rt|jyNr)rHrIXGBGetLastErrorrestyper~callbackXGBRegisterLogCallbackr3)rs r:_register_log_callbackrsN"(//C)+CL !!#,,/143..0115r9c.t}|sy tjdjtj}d}g}|D]} tjj |tjj|gztjd<tjj|}t|dtjj|d} tjj |tjd<n|s4tjj#|d}t%d|d|d t'd t d t(t*t*t*ffd }t-|} |t/} | | k7rQd j d| D} d j d| D} d| d| d|jd} t1| |S#t $rg}YwxYw#t$rU}|jt!|Yd}~tjj |tjd<d}~wwxYw#tjj |tjd<wxYw)zLoad xgboost Library.NPATHFrTrz XGBoost Library (aZ) could not be loaded. Likely causes: * OpenMP runtime is not installed - vcomp140.dll or libgomp-1.dll for Windows - libomp.dylib for Mac OSX - libgomp.so for Linux and other UNIX-like OSes Mac OSX users: Run `brew install libomp` to install OpenMP runtime. * You are running 32-bit Python on a 64-bit OS Error message(s):  verr<c|jddjd\}}}|jd}|dk7r|d|}t|t|t|fS)z(Avoid dependency on packaging (PEP 440).-r.rcroN)rbrtint)rrrrrs r:parsez_load_lib..parsesa"iinQ/55c:ue ZZ  8#2JE5z3u:s5z11r9rc32K|]}t|ywr>rE.0vs r: z_load_lib..s4ec!fec32K|]}t|ywr>rrs r:rz_load_lib..s6v!s1vvrzeMismatched version between the Python package and the native shared object. Python package version: z. Shared object version: z . Shared object is loaded from: z. Likely cause: * XGBoost is first installed with anaconda then upgraded with pip. To fix it please remove one of the installations.)r1renvironrbpathsepKeyErrorrrrrHcdll LoadLibrarysetattrnormpathOSErrorrTrEbasenamer3rrrrr ValueError) lib_paths pathBackup lib_success os_error_listlib_pathrelibnamerlibverpyver pyver_str libver_strrps r: _load_librsHI ZZ'--bjj9 KM ="$bggooh788"BJJv ++))(3C C!1!1(!; <K  "$!=G> I I$I"II""2JretcZ|dk7r&tttjy)zCheck the return value of C API call This function will raise exception when error occurs. Wrap every API call with this function Parameters ---------- ret : return value from API calls rN)r3r0_LIBr)rs r: _check_callrs) ax6$"6"6"89::r9c0|jdd}|r:t|ts*d}|ttft |z }t ||r|j ddk7r td|jdddk(rtd |dd y) z0Validate parameters in distributed environments.deviceNz'Invalid type for the `device` parameterrarozDistributed training doesn't support selecting device ordinal as GPUs are managed by the distributed frameworks. use `device=cuda` or `device=gpu` instead.boostergblinearz booster `z,` is not supported for distributed training.) getrDrErrtyperKrtrNotImplementedError)rZrrps r:_check_distributed_paramsrs ZZ$ 'F j-7 wvtF|,,n &++c"b(   zz)T"j0!y)**V W  1r9 feature_info n_featuresis_column_splitnamect|tst|tstd|dt |t |}t ||k7r(|dk7r#|s!|dd|dt |f}t||S)Nz$Expecting a sequence of strings for , got: rz: must have the same length as the number of data columns, expected , got )rDrErrKrrGrJr)rrrrrps r:_validate_feature_infor4s,$J|X,N24&\@R?S T   %L <J&:??fN O |6#l*;)< = o r9c,tj}ttj tj ||j Jtj|j j}tj|d<|S)a2Build information of XGBoost. The returned value format is not stable. Also, please note that build time dependency is not the same as runtime dependency. For instance, it's possible to build XGBoost with older CUDA version but run it with the lastest one. .. versionadded:: 1.6.0 libxgboost) rHrIrr XGBuildInforrSr\loadsrUr)j_inforWs r: build_inforEsj__ F  f!567 << ## # **V\\((* +C C  Jr9dtypec Ftjtjtjtj tj tjtjtjtjtjtjtji}tjtjur$|tj|tj<||vrtd|j!d|||S)NzSupported types: r)npfloat32rHc_floatfloat64c_doubleuint32c_uintuint64c_uint64int32c_int32int64c_int64intcrKkeys)r_NUMPY_TO_CTYPES_MAPPINGs r:_numpy2ctypes_typerVs FNN FOO 6== 6?? &.. &.. G wwbhh,DRXX,N ) ,, 8 = = ?@w O   $E **r9c^t|jdxr|jjS)N hasobject)hasattrrrr?s r:_array_hasobjectrhs" 4::{ + D 0D0DDr9ct|r td|j}d|vr|dj|d<tt j |d}|S)Nz>#**//4#++a.1H I+,, Jr9ct|tjtjs t dt |}tj|zj |}tj|||s t d|S)z&Convert ctypes pointer to buffer type.zexpected char pointerr)rDrHrc_charr bytearray from_bufferr)rrOrWrptrs r: ctypes2bufferrsi dFNN6==9 :233 F C MMF " / / 4D >>$f -+,, Jr9stringcJtj|jdS)z#Convert a python string to cstring.rC)rHrIencode)rs r:c_strrs ??6==1 22r9rvaluesc t|tjrZ|jjt j |k(r.|j jt j|S|t|z|S)z"Convert a python array to c array.) rDrndarrayritemsizerHsizeofdata_asrrJ)rrs r:c_arrayrsa&"**%&,,*?*?6==QVCW*W}}$$V^^E%:;; ECK & ))r9rcGdd}|}d|vr$||_tj|d}|S||_t j|d}|S)z.Convert array interface to numpy or cupy arrayceZdZUdZdZeeed<edeefdZ e jdeddfdZ edeefdZ e jdeddfd Z y) #from_array_interface..Arrayz3Wrapper type for communicating with numpy and cupy.N _interfacer<c|jSr>)rselfs r:__array_interface__z7from_array_interface..Array.__array_interface__s ?? "r9rcPtj||_t|jd|jd<t|jd|jd<|jjdd&t|jd|jd<yy)Nshaper;r)copyrrcrr rs r:r!z7from_array_interface..Array.__array_interface__s"ii 2DO',T__W-E'FDOOG $&+DOOF,C&DDOOF #""9d3?-24??93M-N *@r9c|jSr>r!rs r:rz.Array.__cuda_array_interface__s++ +r9c||_yr>r'r%s r:rz.Array.__cuda_array_interface__s '0D $r9) r4r5r6r7rrdict__annotations__propertyr!setterrr8r9r:ArrayrsA%) HTN)  #$ #  #  # # O O$ O $ O  ,htn ,  , " ( ( 1d 1t 1 ) 1r9r-streamTr$)rr/arrayr!r)rr-arrouts r:from_array_interfacer3sb114 'C9'0$m!!#D!1 J#,hhs& Jr9ptrr#.is_cudac|r(tjd|}|j}n#tjd|}|j}t j |t jj}ttj|}||dk(sJ||S|df|d<|rd|d<||d <d|d <|S) z2Make an __(cuda)_array_interface__ from a pointer.)r)r#rNrTr;r.r#r) r/emptyrrr!rHrc_void_prSrprod)r4r#rr5r8r0addrrOs r:make_array_interfacer<s  ##$e#<..t51)) ;;sFOO , 2 2D  F  v{* * | 4LE&MhE'NE) Lr9dimspredtsctt||jtjj }t t||tj|}|Sr>) rcrrSrrflattenr3r<r)r#r=r>r5 arr_shaper0s r:_prediction_outputrBsKl5$**bii@HHJKI VY GD E Lr9ceZdZdZ ddeededdfdZdedee e ffdZ e dd Z d e d e de fd Zdd ZddZddZdddefdZeddZede defdZy)DataIteraThe interface for user defined data iterator. The iterator facilitates distributed training, :py:class:`QuantileDMatrix`, and external memory support using :py:class:`DMatrix`. Most of time, users don't need to interact with this class directly. .. note:: The class caches some intermediate results using the `data` input (predictor `X`) as key. Don't repeat the `X` for multiple batches with different meta data (like `label`), make a copy if necessary. Parameters ---------- cache_prefix : Prefix to the cache files, only used in external memory. release_data : Whether the iterator should release the data during iteration. Set it to True if the data transformation (converting data to np.float32 type) is memory intensive. Otherwise, if the transformation is computation intensive then we can keep the cache. N cache_prefix release_datar<cv||_t|_d|_d|_||_d|_d|_y)NF)rE _ProxyDMatrix_handle _exception_enable_categorical_release_temporary_data _data_ref)r rErFs r:__init__zDataIter.__init__s:)$ /3#( $ :>:>r9enable_categoricalcft|dsJdtjdtj|j|_tjtj tj|j|_||_ |j |jfS)zHGet callback functions for iterating in C. This is an internal function.rEz__init__ is not called.N) rrHr|r9_reset_wrapper_reset_callbackr _next_wrapper_next_callbackrK)r rPs r: get_callbackszDataIter.get_callbacksst^,G.GG,Fv//fooF     f.. LL OO    $6 ##T%8%888r9c|jS)zHandle of DMatrix proxy.)rIrs r:proxyzDataIter.proxys||r9fndft_retc|j|S |S#t$r8}tjd}|j ||_Yd}~|Sd}~wwxYw)Nr7)rJ Exceptionsysexc_infowith_traceback)r rYrZrtbs r:_handle_exceptionzDataIter._handle_exception!sZ ?? &N 34K 3"B ..r2DOO 3s A-AAcTd|_|j|j}d|_|y)z.Reraise the exception thrown during iteration.NrMrJ)r excs r:reraisezDataIter.reraise1s0# ?? &//C"DOI 'r9c<|jJ|jJyr>rcrs r:__del__zDataIter.__del__;s$##+++&&&r9thiscb|jrd|_|j|jdy)z,A wrapper for user defined `reset` function.N)rLrMrareset)r rhs r:rRzDataIter._reset_wrapper?s' ==#'D  tzz40r9c tdddddtdttdttdtddf fd j rd_jfd d S) zA wrapper for user defined `next` function. `this` is not used in Python. ctypes can handle `self` of a Python member function automatically when converting it to c function pointer. TN feature_names feature_typesr;rmrnrZr<c~ddlm}m} tj|} j "| | jur j \}}}}n|||| j\}}}}||||f _| j|| jjd||d|| _y#t $rd}YwxYw)Nr)_proxy_transformdispatch_proxy_set_datarlr8) r;rprqweakrefrefrKrMrNrKrXset_info) r;rmrnrZrprqrsnew cat_codesr s r: input_dataz*DataIter._next_wrapper..input_dataOs H kk$'$$0O4>>)?C?S?S<Y }?O!!,, @<Y }%(M=#QD #DJJY ? DJJ   ++   !DN/  sB.. B<;B<c&jSr>)next)rwr sr:z(DataIter._next_wrapper..}sdii .Cr9r)require_keyword_argsr rr%r&rLrMra)r rhrws` @r:rTzDataIter._next_wrapperFs d #5948 ' !' !$L1' !$L1 ' !  ' !  ' ! $' !T ==#'D %%&CQGGr9ct)z>Reset the data iterator. Prototype for user defined function.rrs r:rjzDataIter.resets "##r9rwct)aSet the next batch of data. Parameters ---------- input_data: A function with same data fields like `data`, `label` with `xgboost.DMatrix`. Returns ------- 0 if there's no more batch, otherwise 1. r})r rws r:ryz DataIter.nexts "##r9)NT)r<rHr<N)rhNr<N)r4r5r6r7rrEboolrOrr rVr+rXrrarergrRrrTrrjryr8r9r:rDrDs0HL ?$SM ?@D ?  ? 9 9x?Q9R 9Hrb '17H$7H37Hr$$$x$C$$r9rDerrorcPdtdtfdtdtfffd }|S)amDecorator for methods that issues warnings for positional arguments Using the keyword-only argument syntax in pep 3102, arguments after the * will issue a warning or error when passed as a positional argument. Modified from sklearn utils.validation. Parameters ---------- error : Whether to throw an error or raise a warning. func.r<ctggjjD]c\}}|jtj k(rj |5|jtjk(sSj |etdtdtdtffd }|S)zThrow an error/warning if there are positional arguments after the asterisk. Parameters ---------- f : function to check arguments on. argsrZr<ct|t z } s|dkDr td|dkDrmt d||| dDcgc]\}}| }}}djdj |} r t|t j |tt j|D] \}}|||<  di|Scc}}w)NrzKeyword argument is required.zPass `{}` as keyword args., r8) rJrKzipformatrrurv FutureWarning parameters)rrZ extra_argsr_args_msgrpkargall_argsrr kwonly_argssigs r:inner_fz7require_keyword_args..throw_if..inner_fsTS]2J Q ?@@A~$'{;J'?zklAS#T#TafI#T 399$))H:MN#C.( c=1cnnd33q 4>&> !s C) r ritemskindr POSITIONAL_OR_KEYWORDrT KEYWORD_ONLYrr r)rrparamrrrrrs` @@@r:throw_ifz&require_keyword_args..throw_ifso >>//1KD%zzY<<<%y555""4( 2 t "3 "# "" " "  "(r9)r r)rrs` r:r{r{s0 (xR((Xc2g->(T Or9FceZdZdZdZdZy) DataSplitModez&Supported data split mode for DMatrix.rrN)r4r5r6r7ROWCOLr8r9r:rrs0 C Cr9rc%eZdZdZe d:dddddddddddddej ddedee dee dee d ee d e d ee d ee d eedee dee dee dee dee de deddf"dZdede ddfdZd;dZeddddddddddd dee dee dee dee dee dee dee d ee d ee dee ddfdZdedej,fdZdedej,fdZdede ddfdZdede ddfd Zdede ddfd!ZdDMatrixzData Matrix used in XGBoost. DMatrix is an internal data structure that is used by XGBoost, which is optimized for both memory efficiency and training speed. You can construct DMatrix from multiple different sources of data. NF)weight base_marginmissingsilentrmrnnthreadgroupqidlabel_lower_boundlabel_upper_boundfeature_weightsrPdata_split_moder;labelrrrrrmrnrrrrrrrPrr<c | | td||ntj|_| | nd|_||_t |tjr||_ yddl m }m }||r!|j|||jJy|||j|j||||\}}}|J||_ |j|||| | | | ||||_|||_yy)a Parameters ---------- data : Data source of DMatrix. See :ref:`py-data` for a list of supported input types. label : Label of the training data. weight : Weight for each instance. .. note:: For ranking task, weights are per-group. In ranking task, one weight is assigned to each group (not each data point). This is because we only care about the relative ordering of data points within each group, so it doesn't make sense to assign weights to individual data points. base_margin : Global bias for each instance. See :doc:`/tutorials/intercept` for details. missing : Value in the input data which needs to be present as a missing value. If None, defaults to np.nan. silent : Whether print messages during construction feature_names : Set names for features. feature_types : Set types for features. If `data` is a DataFrame type and passing `enable_categorical=True`, the types will be deduced automatically from the column types. Otherwise, one can pass a list-like input with the same length as number of columns in `data`, with the following possible values: - "c", which represents categorical columns. - "q", which represents numeric columns. - "int", which represents integer columns. - "i", which represents boolean columns. Note that, while categorical types are treated differently from the rest for model fitting purposes, the other types do not influence the generated model, but have effects in other functionalities such as feature importances. For categorical features, the input is assumed to be preprocessed and encoded by the users. The encoding can be done via :py:class:`sklearn.preprocessing.OrdinalEncoder` or pandas dataframe `.cat.codes` method. This is useful when users want to specify categorical features without having to construct a dataframe as input. nthread : Number of threads to use for loading data when parallelization is applicable. If -1, uses maximum threads available on the system. group : Group size for all ranking group. qid : Query ID for data samples, used for ranking. label_lower_bound : Lower bound for survival training. label_upper_bound : Upper bound for survival training. feature_weights : Set feature weights for column sampling. enable_categorical : .. versionadded:: 1.3.0 .. note:: This parameter is experimental Experimental support of specializing for categorical features. If passing 'True' and 'data' is a data frame (from supported libraries such as Pandas, Modin or cuDF), columns of categorical types will automatically be set to be of categorical type (feature_type='c') in the resulting DMatrix. If passing 'False' and 'data' is a data frame with categorical columns, it will result in an error being thrown. If 'data' is not a data frame, this argument is ignored. JSON/UBJSON serialization format is required for this. Nz.Either one of `group` or `qid` should be None.ror)_is_iterdispatch_data_backend)rthreadsrmrnrPr)rrrrrrrr)rrnanrrrrDrHr9handler;rr_init_from_iterrtrmrn)r r;rrrrrrmrnrrrrrrrPrrrrs r:rOzDMatrix.__init__sT  MN N")"5w266 ")"5w2  dFOO ,DK 9 D>  '9 :;;* ** /D LLLL''1+0 , }!!!  #//+    $!.D   $!.D  %r9iteratorc |}|j|j|jr |jndd}tt j |}t j}|j|\}}tjd|jj|||t j|} |jt| ||_ y)N)rrrE)rrrEr@r\r]rHr9rVrXGDMatrixCreateFromCallbackrXrrrer) r rrPitr args_cstrrreset_callback next_callbackrs r:rzDMatrix._init_from_iters ||||/1BOOB  'tzz$'78 "(*(8(89K(L% ..  HHOO    LL   C r9ct|dr9|jJttj |j|`yyNr)rrrr XGDMatrixFreers r:rgzDMatrix.__del__s= 4 ";;* ** **4;;7 8  #r9) rrrrrrrrmrnrc `ddlm} ||j|||j|||j |||j |||j d|||jd|||jd||||_| | |_ | | || dyy) zISet meta info for DMatrix. See doc string for :py:obj:`xgboost.DMatrix`.rdispatch_meta_backendNrrrr)matrixr;r) r;r set_label set_weightset_base_margin set_group set_uint_infoset_float_informrn) r rrrrrrrrmrnrrs r:rtzDMatrix.set_infos 0   NN5 !   OOF #  "   -   NN5 ! ?   uc *  (    35F G  (    35F G  $!.D   $!.D   & !/8I  'r9fieldc dt}tjtj}t t j |jt|tj|tj|t||jtjS)aGet float property from the DMatrix. Parameters ---------- field: str The field name of the information Returns ------- info : array a numpy array of float information of the data )r,rHrrrrXGDMatrixGetFloatInforrrrrSrrr rrOrs r:get_float_infozDMatrix.get_float_infost,fnnV^^,.  & & U5\6<<+?cAR  Crzz::r9c dt}tjtj}t t j |jt|tj|tj|t||jtjS)aGet unsigned integer property from the DMatrix. Parameters ---------- field: str The field name of the information Returns ------- info : array a numpy array of unsigned integer information of the data )r,rHrrrrXGDMatrixGetUIntInforrrrrSrrrs r: get_uint_infozDMatrix.get_uint_infost+fnnV]]+-  % % U5\6<<+?cAR  Cryy99r9c&ddlm}||||dy)zSet float type property into the DMatrix. Parameters ---------- field: str The field name of the information data: numpy array The array of data to be set rrrdNr;rr rr;rs r:rzDMatrix.set_float_infos 0dD%9r9c&ddlm}||||dy)aSet float type property into the DMatrix for numpy 2d array input Parameters ---------- field: str The field name of the information data: numpy array The array of data to be set rrrdNrrs r:set_float_info_npy2dzDMatrix.set_float_info_npy2d s 0dD%9r9c&ddlm}||||dy)zSet uint type property into the DMatrix. Parameters ---------- field: str The field name of the information data: numpy array The array of data to be set rrrNrrs r:rzDMatrix.set_uint_infos 0dD%:r9fnamec tjtjj|}t t j |jt|tj|y)ajSave DMatrix to an XGBoost buffer. Saved binary can be later loaded by providing the path to :py:func:`xgboost.DMatrix` as input. Parameters ---------- fname : string or os.PathLike Name of the output buffer file. silent : bool (optional; default: True) If set, the output is suppressed. N) rfspathr expanduserrrXGDMatrixSaveBinaryrrrHr)r rrs r: save_binaryzDMatrix.save_binary*sL "'',,U34  $ $T[[%, V@T U r9c&ddlm}|||ddy)zSet label of dmatrix Parameters ---------- label: array like The label information to be set into DMatrix rrrrdNr)r rrs r:rzDMatrix.set_label:s 0dE7Gr9marginc&ddlm}|||ddy)aSet base margin of booster to start from. This can be used to specify a prediction value of existing model to be base_margin However, remember margin is needed, instead of transformed prediction e.g. for logistic regression: need to put in value before logistic transformation see also example/demo.py Parameters ---------- margin: array like Prediction margin of each datapoint rrrrdNr)r rrs r:rzDMatrix.set_base_marginZs 0dFM7Cr9c&ddlm}|||ddy)zSet group size of DMatrix (used for ranking). Parameters ---------- group : array like Group size of each group rrrrNr)r rrs r:rzDMatrix.set_groupls 0dE7H=r9c$|jdS)z]Get the label of the DMatrix. Returns ------- label : array rrrs r: get_labelzDMatrix.get_labelxs""7++r9c$|jdS)z_Get the weight of the DMatrix. Returns ------- weight : array rrrs r: get_weightzDMatrix.get_weights""8,,r9c$|jdS)zaGet the base margin of the DMatrix. Returns ------- base_margin rrrs r:get_base_marginzDMatrix.get_base_margins""=11r9cN|jd}tj|S)zUGet the group of the DMatrix. Returns ------- group group_ptr)rrdiff)r rs r: get_groupzDMatrix.get_groups#&&{3 wwy!!r9c tj|jdztj}tj|j tj }tj|j tj }|jjtjt}|jjtjtj}|jjtjtj}ttji}t!t"j%|j&||||t(j*j-|||f|j|j/f}|S)zGet the predictors from DMatrix as a CSR matrix. This getter is mostly for testing purposes. If this is a quantized DMatrix then quantized values are returned instead of input values. .. versionadded:: 1.7.0 rr)r#)rr8num_rowrnum_nonmissingrrrHrrr,c_uint32rr@r\r]rrXGDMatrixGetDataAsCSRrscipysparse csr_matrixnum_col) r indptrindicesr;c_indptr c_indicesc_dataconfigrs r:get_datazDMatrix.get_datas0$,,.1,BII>((4..0 Bxx++-RZZ@==(( )DENN**6>>&//+JK $$V^^FNN%CD#DJJrN3  & &t{{FHiQW X ll%% 7F #DLLNDLLN+K&  r9c |j}tj}tj}t}t t j |j|tj|tj||jJ|jJtj|j}t|}|j|dzk(sJ|jtj k(sJtj|j}t|}|j|dk(sJ|jtj"k(sJ||fS)zNGet quantile cuts for quantization. .. versionadded:: 2.0.0 rro)rrHrIr^rrXGDMatrixGetQuantileCutrrrSr\rr3sizerrrr) r r c_sindptrc_sdatari_indptrri_datar;s r:get_quantile_cutzDMatrix.get_quantile_cuts \\^ OO% //#  ( ( VV\\)%S  ***}}(((::ioo.%h/{{j1n,,,||ryy(((GMM*#F+yyF2J&&&zzRZZ'''t|r9ct}ttj|jt j ||jS)z&Get the number of rows in the DMatrix.)r,rrXGDMatrixNumRowrrHrrSr rs r:rzDMatrix.num_row5mD((fll36GHIyyr9ct}ttj|jt j ||jS)z4Get the number of columns (features) in the DMatrix.)r,rrXGDMatrixNumColrrHrrSrs r:rzDMatrix.num_colrr9ct}ttj|jt j ||jS)z_Get the number of non-missing values in the DMatrix. .. versionadded:: 1.7.0 )r,rrXGDMatrixNumNonMissingrrHrrSrs r:rzDMatrix.num_nonmissings7 mD// V\\#=NOPyyr9ct}ttj|jt j |t|jS)zRGet the data split mode of the DMatrix. .. versionadded:: 2.1.0 ) r,rrXGDMatrixDataSplitModerrHrrrSrs r:rzDMatrix.data_split_modes< mD// V\\#=NOPSYY''r9rindex allow_groupsc |ddlm}tj}||tj }t tj|jttj|tt|tj|tj|rdndt|S)anSlice the DMatrix and return a new DMatrix that only contains `rindex`. Parameters ---------- rindex List of indices to be selected. allow_groups Allow slicing of a matrix with a groups attribute Returns ------- res A new DMatrix containing only selected indices. r)_maybe_np_slicerr)r;rrHr9rrrrXGDMatrixSliceDMatrixExrrrr,rJrr)r rrrrs r:slicez DMatrix.slices" *" rxx8  ( (  f-CK( V$ ,QA6   vr9c <t}tjtj}t t j |jtdtj|tj|t||}|sy|S)zmLabels for features (column labels). Setting it to ``None`` resets existing feature names. feature_nameN r,rHrrIrrXGDMatrixGetStrFeatureInforrrrY)r rOsarrrms r:rmzDMatrix.feature_namessy.v~~foo.0  + + n% V$ T"   +48 r9c "|>ttj|jt ddt dyt ||j|jtjk(d}t|tt|k7rNtj|ddd\}}t||Dcgc] \}}|dkDs |}}}t!d|t#d |Ds t!d |Dcgc]}t%|d }}t'j(t|z|} ttj|jt d| t t|ycc}}wcc}w) Nrrz feature namesFT) return_indexreturn_inverse return_countsrz0feature_names must be unique. Duplicates found: c3lK|]+ttxrtfddD -yw)c3&K|]}|v ywr>r8)rxrs r:rz2DMatrix.feature_names...Js*K?a16?s)[].Is6 " q#  Ks*K?*K'K#K K"s14z;feature_names must be string, and may not contain [, ] or ttj|jt ddt dyt ||j|jtjk(d}|Dcgc]}t|d}}tjt|z|}ttj|jt d|t t|ycc}w)Nr.rz feature typesrCr)rrr%rrr,rrrrrrFrHrIrJ)r rnrfeature_types_bytesc_feature_typess r:rnzDMatrix.feature_typesws  //KK~!6k!n  .  LLN  "m&7&7 7   DQQ=auQ9=Q!??S1D-EE     + + n%C ./   RsC;r>r)T)F)8r4r5r6r7_deprecate_positional_argsrrr#rrrdrr%r&rrOrDrrgrtrErrrrrrrrrPathLikerrrrrrrrrrrrrrrrrrrrrr+rmr,rnr8r9r:rrs &*V/ '++/#'0404!%%)#'1515/3#()6):):%V/V/ "V/ # V/ i( V/%V/V/ -V/ -V/#V/ "V/i V/$I.V/$I.V/ "),!V/"!#V/$'%V/& 'V/ V/pdt.  &*&*+/%)#'15150404/3& "&# & i( &  " &i &$I.&$I.& -& -&"),& & &P;C;BJJ;,:3:2:::, :C :y :T ::#:Y:4: ;3 ;i ;D ; sBKK'7!8 $ RV =y =T =??t?(DiDDD$ >y >T >,2::,-BJJ-22"2::"%,,112% BJJ(>"?<  ((JODIrzz12BF Bx 5*0 8L+A0 d0 0 dx 50 8L+A d  r9rceZdZdZd dZdeddfdZdededdfdZde jddfd Z deddfd Z d e jjddfd Zy)rHzdA placeholder class when DMatrix cannot be constructed (QuantileDMatrix, inplace_predict). r<Nctj|_ttj tj |jyr>)rHr9rrrXGProxyDMatrixCreaterrs r:rOz_ProxyDMatrix.__init__s0oo' D--fll4;;.GHIr9r;c|j}ttj|d}t t j |j|y)z)Reference data from CUDA array interface.rCN)rrFr\r]rr'XGProxyDMatrixSetDataCudaArrayInterfacer)r r;rrs r:_ref_data_from_cuda_interfacez+_ProxyDMatrix._ref_data_from_cuda_interfaces>11 djj3W=   8 8m T r9rvctddlm}|||}ttj |j |y)z)Reference data from CUDA columnar format.r)_cudf_array_interfacesN)r;r;rr!XGProxyDMatrixSetDataCudaColumnarr)r r;rvr;interfaces_strs r:_ref_data_from_cuda_columnarz*_ProxyDMatrix._ref_data_from_cuda_columnars*0/i@D::4;;WXr9cnddlm}ttj |j ||y)z Reference data from numpy array.r_array_interfaceN)r;rArrXGProxyDMatrixSetDataDenser)r r;rAs r:_ref_data_from_arrayz"_ProxyDMatrix._ref_data_from_arrays'*  + +DKK9I$9O P r9crttj|j|j y)zRReference data from a pandas DataFrame. The input is a PandasTransformed instance.N)rrXGProxyDMatrixSetDataColumnarrarray_interface)r r;s r:_ref_data_from_pandasz#_ProxyDMatrix._ref_data_from_pandass'  . .t{{DrrrCrGrrrrLr8r9r:rHrHs J ( t YYdYtY    ( t  ell&=&=  $  r9rHc) eZdZdZe ddddddddddddddddej ddedee dee dee d ee d e d ee d ee d eedeedeedee dee dee dee dee de deddf&dZdedeede deddf dZy)QuantileDMatrixacA DMatrix variant that generates quantilized data directly from input for the ``hist`` tree method. This DMatrix is primarily designed to save memory in training by avoiding intermediate storage. Set ``max_bin`` to control the number of bins during quantisation, which should be consistent with the training parameter ``max_bin``. When ``QuantileDMatrix`` is used for validation/test dataset, ``ref`` should be another ``QuantileDMatrix``(or ``DMatrix``, but not recommended as it defeats the purpose of saving memory) constructed from training dataset. See :py:obj:`xgboost.DMatrix` for documents on meta info. .. note:: Do not use ``QuantileDMatrix`` as validation/test dataset without supplying a reference (the training dataset) ``QuantileDMatrix`` using ``ref`` as some information may be lost in quantisation. .. versionadded:: 1.7.0 Parameters ---------- max_bin : The number of histogram bin, should be consistent with the training parameter ``max_bin``. ref : The training dataset that provides quantile information, needed when creating validation/test dataset with ``QuantileDMatrix``. Supplying the training DMatrix as a reference means that the same quantisation applied to the training data is applied to the validation/test data NF)rrrrrmrnrmax_binrsrrrrrrPrr;rrrrrrmrnrrOrsrrrrrrPrr<ct| |_||ntj|_| | nd|_||_t |tjr||_ y| | tdt |tr'td|||||| | |||f Dr td|j|| |||| | |||||| y)NrozVOnly one of the eval_qid or eval_group for each evaluation dataset should be provided.c3$K|]}|du ywr>r8)rinfos r:rz+QuantileDMatrix.__init__..s! DD  szYIf data iterator is used as input, data like label should be specified as batch argument.) rsrrrrrrrrrmrnrP)rOrrrr_silentrDrHr9rrrDr$_init)r r;rrrrrrmrnrrOrsrrrrrrPrs r:rOzQuantileDMatrix.__init__s. ")"5w266 ")"5w2  dFOO ,DK  ?u0.  dH %!!%%# !3 #//+''1  r9metac ddlm}m}m}m}||r||}||r|} n |dd|i|} t j } | j|\} } | j tdt|j|j|j} tjd| j j"| |j"n|| | | t j$| }| j't)|| |_y)Nr)SingleBatchInternalIter _is_dlpackr_transform_dlpackr;zZQuantileDMatrix doesn't cache data, remove the cache_prefix in iterator to fix this error.)rrrOr8)r;rWrXrrYrHr9rVrErr^rrrOr#XGQuantileDMatrixCreateFromCallbackrXrrrer)r r;rsrPrUrWrXrrYrrrrrrs r:rTzQuantileDMatrix._init5s   d %T*D D>B(;d;d;B"(*(8(89K(L% ?? &1  LL$,,  66  HHOO/CJJs    LL   C r9r>)r4r5r6r7r2rrr#rrrdrr%r&rrrOr rTr8r9r:rNrNs> &*G '++/#'0404!%!%!%%)#'1515/3#()6):):)G G  "G # G i( G %G G  -G  -G #G #G g G  "G i G $I.!G "$I.#G $"),%G &!'G (')G * +G  G R..g .! .  .  .r9rNc0eZdZdZdededdffd ZxZS)DeviceQuantileDMatrixz\Use `QuantileDMatrix` instead. .. deprecated:: 1.7.0 .. versionadded:: 1.1.0 rrZr<NcXtjdtt||i|y)Nz%Please use `QuantileDMatrix` instead.)rurvrsuperrO)r rrZ __class__s r:rOzDeviceQuantileDMatrix.__init__os$ =}M $)&)r9)r4r5r6r7r rO __classcell__)r_s@r:r\r\fs'*c*S*T**r9r\paramsct|trXd|vrTt|dtrA|d}|jddt|j }|D] }|d|fgz } |S|S)N eval_metric)rDr)rGpopr)ra eval_metrics params_listrcs r:_configure_metricsrgxsr64 V # vm,d 3m,  =$'6<<>* 'K ]K89 9K( Mr9ceZdZdZ dedeedeeedeede e je fddfdZ deee efe eed ffdeeed fe ffd Zdeeee e fdee eeeffd Zdedefd Zdfd ZdefdZdeddfdZdeeeefddfdZdedfdZde fdZde ddfdZdgdZ de!ddfdZ"dgdZ#de dee fdZ$dee ee ffdZ%dee!ddfd Z&d!e dee'fd"Z(d#ee'd!e ddfd$Z)e*dee+fd%Z,e,jZd#ee+ddfd&Z,e*dee.fd'Z/e/jZd#ee.ddfd(Z/ dhdeee0ee e!fe fdee ddfd)Z1 dhd*ed+ed,ee2ddfd-Z3d*ed+ed.e4d/e4ddf d0Z5 did2eeee fd+ed3ee6d4e7de f d5Z8djd6ed7e d+ede fd8Z9 dkd6ed4e7d;e7de7d?e7d@e7dAe:dBe7de;jxfdCZ=d:de;j|d1dd9fd6e?dAe:dDe dEe@d?e7dFe!dBe7de4fdGZAdHee e jfddfdIZBdldJe de fdKZCdHeDddfdLZEe*defdMZFeFjZd+eddfdNZFe*de@fdOZGeGjZdPeddfdQZGdefdRZHdefdSZI dmdTee e jfdUee e jfdVe7dWe ddf dXZJ dmdUee e jfdVe7dWe dee fdYZK dndUee e jfdee ee@ee@fffdZZL dodUee e jfd[e dee ee@ee@fffd\ZMdndUee e jfdeNfd]ZOd6eddfd^ZPd_ee.ddfd`ZQ dpdae dUee je fdbeedce7dee;jxeNff ddZRy)qBoosterzA Booster of XGBoost. Booster is the model of xgboost, that contains low level routines for training, prediction and evaluation. Nracache model_filer<c ||ng}|D]4}t|trtdt|j|t t j|Dcgc]}|jc}}t j|_ttj|tt|t j|j|D]}|j|t|t r|jJ|j#}|d}|d=t j$t|zj'|}tt|} ttj)|j|| |j*j-|nFt|t.t0j2t4fr|j7|n|n td||xsi}t9|j;} |j=| } t| t>r| jAdnd| d<|jC| xsiycc}w)z Parameters ---------- params : Parameters for boosters. cache : List of cache items. model_file : Path to the model file if it's string or PathLike. Nzinvalid cache item: rz Unknown type:)validate_parametersTTrm)"rDrrKrr4rrHr9rrrXGBoosterCreater,rJr_assign_dmatrix_featuresri __getstate__r r XGBoosterUnserializeFromBuffer__dict__updaterErr3r  load_modelrgr$_configure_constraintsrGrT set_param) r rarjrkrLdmatsstaterr4rOparams_processeds r:rOzBooster.__init__s"*Aa)"6tAw7G7G6H I5QQE)BEq!((E)BC171B   {3u:. T[[0I  A  ) )! , j' *;;* **++-E8_Fh==3v;.;;FCC V-F ;;DKKfU V MM  '  S"++y$A B OOJ '   OZ8 82-fkkm<667GH & -  # #$A B6: 2 3 '-2.K*CsI rS.cttrSttrStj }|j xsg}|j t|s tdtfd|DS)NDConstrained features are not a subset of training data feature namesc3BK|]}j|dyw)rN)r)rrrSs r:rz9Booster._transform_monotone_constrains..sBMDUYYtQ'Ms)rDrErcr&rrmissubsetr)r rSconstrained_featuresrms ` r:_transform_monotone_constrainsz&Booster._transform_monotone_constrainssz eS !L eU #L"5::<0**0b #,,S-?@V BMBBBr9c*t|tr|St|jxsgDcic]\}}|| }}} g}|D]$}|j |Dcgc]}|| c}&|Scc}}wcc}w#t $r}t d|d}~wwxYw)Nr{)rDrE enumeratermrTrr) r rSidxrfeature_idx_mappingr_ constraintrrs r:"_transform_interaction_constraintsz*Booster._transform_interaction_constraintss eS !L'01C1C1Ir'J 'J)#tD#I'J   F#  KUV:<(6:V$M W V  s/ A-A8 A3 " A83A88 B B  Bct|trt|j}t |D]E\}}|\}}||dk(r||j |f||<*|dk(s0||j |f||<G|S)Nmonotone_constraintsinteraction_constraints)rDr)rGrrrr)r rarrrrSs r:ruzBooster._configure_constraintss fd #&,,.)F#F+JCKD%}--#T%H%H%OPs 22#T%L%LU%STs , r9ct|dr=|j0ttj |jd|_yyyr)rrrr XGBoosterFreers r:rgzBooster.__del__s; 4 "t{{'> **4;;7 8DK(? "r9c |jj}|d}|t}tjtj }t tj|jtj|tj|t||j}||d<|Sr) rrr$r,rHrr rrXGBoosterSerializeToBufferrrrrS)r rhrrOrbufs r:rpzBooster.__getstate__s}}!!#h   ]F06>>&--02D //KKf!5v||D7I   fll3C DN r9rxc |d}||}ttjg}tj}ttj |t dtj|t t|}tjt|zj|}ttj|||||d<|jj|y)Nrr)rrHr9rrrnr,rrJr r rqrrrs)r rxrrrwrOr4s r: __setstate__zBooster.__setstate__sx  CFOOR0E__&F $$UKNFLL *1+C c5 !IJ J#u%3rzz4>BDINCC. ciih 0CII4EEIIE chhX /3883CD88De| #6688/sxxQ,,u%d#d#) $$ KK&&&,,}2M  R<78 8FD&&v}}56%  r9)riNNc#XKtd|jD] }|| yw)zXIterator method for getting individual trees. .. versionadded:: 2.0.0 rN)rRnum_boosted_rounds)r rXs r:__iter__zBooster.__iter__Ys+ q$1134Aq'M5s(*c 2tj}t}ttj |j tj|tj||jJ|jj}|S)zwOutput internal parameter configuration of Booster as a JSON string. .. versionadded:: 1.0.0 ) rHrIr,rrXGBoosterSaveJsonConfigrrrSrU)r  json_stringrOr_s r: save_configzBooster.save_configbswoo'   ( ( V\\&16<< 3L    ,,,""))+ r9rct|tsJttj |j t |y)zWLoad configuration returned by `save_config`. .. versionadded:: 1.0.0 N)rDrErrXGBoosterLoadJsonConfigrr)r rs r: load_configzBooster.load_configts1 &#&&&D00eFmLMr9c$|jdSr>) __deepcopy__rs r:__copy__zBooster.__copy__|s  &&r9rct|S)zReturn a copy of booster.)rk)ri)r rs r:rzBooster.__deepcopy__s $''r9c,tj|S)zwCopy the booster object. Returns ------- booster : A copied booster model r/rs r:r$z Booster.copysyyr9keyc \tj}tj}ttj |j t|tj|tj||jdk7r|j}|sJt|Sy)aGet attribute string from the Booster. Parameters ---------- key : The key to get attribute from. Returns ------- value : The attribute value of the key, returns None if attribute do not exist. rN) rHrIrrrXGBoosterGetAttrrrrrSr0)r rrsuccessrSs r:attrz Booster.attrsoo,,.  ! ! U3Zc):FLL.is_flattenXs(u{{#q(?EKKNa,? ?r9cdt|}t|tjs|s t | j }|j d|k7rA |r9tjdt|j||j|z}t|tjr ||j\}}|}|S|r |}|St |)NzHExpecting `np.ndarray` or `cupy.ndarray` for gradient and hessian. Got: rzvSince 2.1.0, the shape of the gradient and hessian is required to be (n_samples, n_targets) or (n_samples, n_classes).) rrDrrrKrr#rurvrreshaperr) r0rp n_samplesrrrArrrrrs r:rFz&Booster.boost..array_interface[se ' eRZZ09Nn$(I{{1~*z%/@ L!  iy1HI%,+E5;;?q,U3    &1%8   n$r9N) r;rArrrror*rrFrrXGBoosterTrainOneIterr) r rrrrrFrArrrrs ` @@@@@r:rz Booster.boost>s"   %%f- @k @d @ ; 5  8   & &  %%   r9Tevalsfevalrc |D]}t|dts$tdt|djt|dt s$tdt|dj|j |dttj|Dcgc]}|djc}}ttj|Dcgc]}t|dc}}tj}ttj|jtj |||t#t%|tj&||j(J|j(j+} |[|D]V\} } ||j-| d|| } t| t.r| D]\} }| d| | |fzz } G| \} }| d| | |fzz } X| Scc}wcc}w)aHEvaluate a set of data. Parameters ---------- evals : List of items to be evaluated. iteration : Current iteration. feval : Custom evaluation function. Returns ------- result: str Evaluation result string. rzexpected DMatrix, got rzexpected string, got F)rrz %s-%s:%f)rDrrKrr4rErorrHr9rrIrrrXGBoosterEvalOneIterrr,rJrrSrUrrG)r rrrrrLrwevnamesrprWdmatevname feval_retrrs r:eval_setzBooster.eval_sets0AadG,"8ad9L9L8M NOOadC("7QqT 8K8K7L MNN  ) )!A$ / u)Eu!!A$++u)EF&//+GAE!A$K+GHoo  % %  Y'CJ' S!   yy$$$ii    % f!LL]LS i.%. c|vtS.AAA&/!*ID#<64*===C!& ;*F+Gs "G3 G8 r;rcN|j||j||fg|S)aSEvaluate the model on mat. Parameters ---------- data : The dmatrix storing the input. name : The name of the dataset. iteration : The current iteration number. Returns ------- result: str Evaluation result string. )ror)r r;rrs r:evalz Booster.evals)& %%d+}}tTl^Y77r9Frr pred_leaf pred_contribsapprox_contribspred_interactionsvalidate_featuresriteration_range strict_shapec t|tstdt||r|j} |j | d|t | dt | d| ddt ddffd } |r| d|r | |sd nd |r | |sd nd |r| d tjtj} tjt}t}ttj|j|jtt!j"tj$|tj$|tj$| t'||| dS)a*Predict with data. The full model will be used unless `iteration_range` is specified, meaning user have to either slice the model or use the ``best_iteration`` attribute to get prediction from best model returned from early stopping. .. note:: See :doc:`Prediction ` for issues like thread safety and a summary of outputs from this function. Parameters ---------- data : The dmatrix storing the input. output_margin : Whether to output the raw untransformed margin value. pred_leaf : When this option is on, the output will be a matrix of (nsample, ntrees) with each record indicating the predicted leaf index of each sample in each tree. Note that the leaf index of a tree is unique per tree, so you may find leaf 1 in both tree 1 and tree 0. pred_contribs : When this is True the output will be a matrix of size (nsample, nfeats + 1) with each record indicating the feature contributions (SHAP values) for that prediction. The sum of all feature contributions is equal to the raw untransformed margin value of the prediction. Note the final column is the bias term. approx_contribs : Approximate the contributions of each feature. Used when ``pred_contribs`` or ``pred_interactions`` is set to True. Changing the default of this parameter (False) is not recommended. pred_interactions : When this is True the output will be a matrix of size (nsample, nfeats + 1, nfeats + 1) indicating the SHAP interaction values for each pair of features. The sum of each row (or column) of the interaction values equals the corresponding SHAP value (from pred_contribs), and the sum of the entire matrix equals the raw untransformed margin value of the prediction. Note the last row and column correspond to the bias term. validate_features : When this is True, validate that the Booster's and data's feature_names are identical. Otherwise, it is assumed that the feature_names are the same. training : Whether the prediction value is used for training. This can effect `dart` booster, which performs dropouts during training iterations but use all trees for inference. If you want to obtain result with dropouts, set this parameter to `True`. Also, the parameter is set to true when obtaining prediction for custom objective function. .. versionadded:: 1.0.0 iteration_range : Specifies which layer of trees are used in prediction. For example, if a random forest is trained with 100 rounds. Specifying `iteration_range=(10, 20)`, then only the forests built during [10, 20) (half open set) rounds are used in this prediction. .. versionadded:: 1.4.0 strict_shape : When set to True, output shape is invariant to whether classification is used. For both value and margin prediction, the output shape is (n_samples, n_groups), n_groups == 1 when multi-class is not used. Default to False, in which case the output shape can be (n_samples, ) if multi-class is not used. .. versionadded:: 1.4.0 Returns ------- prediction : numpy array z,Expecting data to be a DMatrix object, got: rr)rriteration_begin iteration_endrrqr<Nc6ddk7r td|d<y)Nrrz!One type of prediction at a time.)r)rqrs r: assign_typez$Booster.predict..assign_type@ s$F|q  !DEEDLr9r7F)rDrrKrrm_validate_featuresrrHrrr,rrXGBoosterPredictFromDMatrixrr@r\r]rrB)r r;rrrrrrrrrrYrpredsr#r=rs @r:rzBooster.predictsMz$(JDQUJW W ##B  # #B ' "?1#56 !34(   3 4   N  a 8  a 8  N.v~~.0+{+-}  , ,  "4::d#34 U# T" U#   "%ue<Yt_88D>D 4)U4::!#(9(9(;tzz!}(L 89J9J9L8MN::a=/+ T " .&tTZZ8GD! ..KK$T*LL'LL&LL' &eT5%@ @ d- . 11KK((*LL'LL&LL' &eT5%@ @ dELL33 4 4)$5D ,,KK$T[[1$T\\2$TYY/ 1 .LL'LL&LL'  &eT5%@ @ $  3(.D1$7M 22KK!LL'LL&LL' &eT5$? ? t  H&8dD"4' #D)S!4D)DN '', 55KK"LL'LL&LL' &eT5$? ? 3tDz? *-T T  r9rc"t|ttjfretjtj j |}ttj|jt|ytd)aESave the model to a file. The model is saved in an XGBoost internal format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature_names) are only saved when using JSON or UBJSON (default) format. See :doc:`Model IO ` for more info. .. code-block:: python model.save_model("model.json") # or model.save_model("model.ubj") Parameters ---------- fname : Output file name z%fname must be a string or os PathLikeN) rDrErr3rrrrrXGBoosterSaveModelrrrK)r rs r: save_modelzBooster.save_model< s]( ec2;;/ 0IIbgg0078E // U5\J KCD Dr9 raw_formatc Lt}tjtj}t |}t t j|j|tj|tj|t||jS)azSave the model to a in memory buffer representation instead of file. The model is saved in an XGBoost internal format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature_names) are only saved when using JSON or UBJSON (default) format. See :doc:`Model IO ` for more info. Parameters ---------- raw_format : Format of output buffer. Can be `json`, `ubj` or `deprecated`. Returns ------- An in memory buffer representation of the model )r) r,rHrr r^rrXGBoosterSaveModelToBufferrrrrS)r r&rOrrs r:save_rawzBooster.save_rawV st",v~~fmm,.J/  + + VV\\&%96<<;M  T6<<00r9ct|ttjfretjtj j |}ttj|jt|yt|trl|}tt|}tj t|zj#|}ttj%|j||yt'd|)ayLoad the model from a file or a bytearray. The model is saved in an XGBoost internal format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature_names) are only saved when using JSON or UBJSON (default) format. See :doc:`Model IO ` for more info. .. code-block:: python model.load_model("model.json") # or model.load_model("model.ubj") Parameters ---------- fname : Input file name or memory buffer(see also save_raw) zUnknown file type: N)rDrErr3rrrrrXGBoosterLoadModelrrr r,rJrHr r XGBoosterLoadModelFromBufferrK)r rrrOr4s r:rtzBooster.load_modelq s( ec2;;/ 0IIbgg0078E // U5\J K y )C S*F==3s8+88=C 99$++sFS T159 9r9cT|jd}| t|Std)z#The best iteration during training.best_iterationz=`best_iteration` is only defined when early stopping is used.)rrAttributeErrorr bests r:r.zBooster.best_iteration s3yy)*  t9  K  r9c(|j|y)N)r.r)r rs r:r.zBooster.best_iteration s Y /r9cT|jd}| t|Std)z*The best evaluation score during training. best_scorez9`best_score` is only defined when early stopping is used.)rrdr/r0s r:r5zBooster.best_score s2yy&  ;  G  r9scorec(|j|y)N)r5r3)r r6s r:r5zBooster.best_score s  'r9ctj}|jJttj |jtj ||jS)zmGet number of boosted rounds. For gblinear this is reset to 0 after serializing the model. )rHrrrrXGBoosterBoostedRoundsrrS)r roundss r:rzBooster.num_boosted_rounds sK {{&&&D// V\\&=QRS||r9ct}|jJttj |jt j ||jS)zNumber of features in booster.)r,rrrXGBoosterGetNumFeaturerHrrSrs r:rzBooster.num_features sE={{&&&D// V\\(=STU~~r9foutfmap with_stats dump_formatcnt|ttjfrCtjtj j |}t|dd}d}n|}d}|j|||}|dk(rj|jdt|D]9\}} |j| |t|dz ks)|jd ;|jd n9t|D]+\}} |jd |d |j| -|r|jy y )a1Dump model into a text or JSON file. Unlike :py:meth:`save_model`, the output format is primarily used for visualization or interpretation, hence it's more human readable but cannot be loaded back to XGBoost. Parameters ---------- fout : Output file name. fmap : Name of the file containing feature map names. with_stats : Controls whether the split statistics are output. dump_format : Format of model dump file. Can be 'text' or 'json'. wrCrTFr\z[ rz, z ]zbooster[z]: N) rDrErr3rrrrget_dumpwriterrJclose) r r=r>r?r@fout_obj need_closerrXrs r: dump_modelzBooster.dump_model s, dS"++. /99RWW//56DD#8HJHJmmD*k: & NN5 !#C.3s#s3x!|#NN5)) NN5 !#C.3!D12s#)  NN  r9c tjtjj|}t }t j t j}ttj|jt|t j|t|t j|t j|t||}|S)aReturns the model dump as a list of strings. Unlike :py:meth:`save_model`, the output format is primarily used for visualization or interpretation, hence it's more human readable but cannot be loaded back to XGBoost. Parameters ---------- fmap : Name of the file containing feature map names. with_stats : Controls whether the split statistics are output. dump_format : Format of model dump. Can be 'text', 'json' or 'dot'. )rrrrr,rHrrIrrXGBoosterDumpModelExrrrrrY)r r>r?r@rOrrWs r:rCzBooster.get_dump s(yy++D12.v~~foo.0  % % d  Z(k" V$ T"   !v. r9c(|j|dS)auGet feature importance of each feature. .. note:: Zero-importance features will not be included Keep in mind that this function does not include zero-importance feature, i.e. those features that have not been used in any split conditions. Parameters ---------- fmap : The name of feature map file r)importance_type) get_score)r r>s r: get_fscorezBooster.get_fscore s ~~dH~==r9rLc tjtjj|}t j tj }t j tj}t}t}t j t}ttj|jt||t j|t j|t j|t j|t j|t||}t!|||d} i} t#| j$dkDrD| j$ddkDr2t'|| D]!\} } | D cgc] } t)| c} | | <#| St'|| D]\} } t)| | | <| Scc} w)a'Get feature importance of each feature. For tree model Importance type can be defined as: * 'weight': the number of times a feature is used to split the data across all trees. * 'gain': the average gain across all splits the feature is used in. * 'cover': the average coverage across all splits the feature is used in. * 'total_gain': the total gain across all splits the feature is used in. * 'total_cover': the total coverage across all splits the feature is used in. .. note:: For linear model, only "weight" is defined and it's the normalized coefficients without bias. .. note:: Zero-importance features will not be included Keep in mind that this function does not include zero-importance feature, i.e. those features that have not been used in any split conditions. Parameters ---------- fmap : The name of feature map file. importance_type : One of the importance types defined above. Returns ------- A map between feature names and their scores. When `gblinear` is used for multi-class classification the scores for each feature is a list with length `n_classes`, otherwise they're scalars. )rL feature_mapFr)rrrrrHrrIrr,rrXGBoosterFeatureScorerr^rrYrBrJr#rrd)r r>rLrscoresn_out_featuresout_dimr# features_arr scores_arrresultsfeatr6rfs r:rMzBooster.get_score$ sFyy++D1226>>&//24//1$-+{+-  & & ON ^, X& W% U# V$  *(NC 'wF 8: z 1 $)9)9!)dartgbtreez,This method is not defined for Booster type T)r?rr!rrrraz=|,z\b\d+\bLeafNANrr"r#roNz:{,z Failed to parse model text dump.rr ) TreeNodeIDFeatureSplitYesNoMissingGainCoverCategory sort_valuesrcrd)drop)rrrrr- ImportErrorr\rrrrCrrbrJrerTrfindallrdrtrErr.callablegetattrrn reset_indexsort)r r>rtree_idsnode_idsfidssplits categories y_directs n_directsmissingsgainscoverstreesrXtreeliner1rstatsfidcats cats_splitstr_it_idn_ididsdfs r:trees_to_dataframezBooster.trees_to_dataframef sVyy++D12;  **T--/0;@ -/ -/ ,. dt 4 'GAt 4(jjos8q=rd{ FLL-EHHUE!H5EOOA&OOC :uQx(H(K$LMKK'MM%,/%%eEl3$$U5\2$$U5\2OOE%L1LLuQx1MM%a/2a&,,s+C1v{{3'2- #A S 1 eE!Ho6"))$/QT*b0 #A S 1$Qx"~%)ZZ_  eEl3")) *M()KLLHHUCF3EOOA&OOC :s1v(Fq(I$JKKKa)FE$$US[58%;<$$US[58%;<OOECK%($:;LLuQx1MM%a/2g)(l>A8=TU=TztTs4y3T*=TU    #&   GB t4 5>>66"23??T?J Jww'(44$4??+Vs?$Sc|jdk(ry|j}|j}|j||_|j||_|j|yr)rrmrnr)r r;rYfts r:roz Booster._assign_dmatrix_features s_ <<>Q            %!#D     %!#D  #r9rmcn|jy|3|j'tddj|jz|j|k7rtt t |jtt t |z }tt t |tt t |jz }d}|r!|ddjd|Dzdzz }|r|ddjd|Dzz }t|j |j|y) Nz1training data did not have the following fields: rzfeature_names mismatch: {0} {1}z expected c32K|]}t|ywr>rrrfs r:rz-Booster._validate_features.. s< 1A rz in input dataz2 training data did not have the following fields: c32K|]}t|ywr>rrs r:rz-Booster._validate_features.. s; 1A r)rmrrr&rr%r)r rm dat_missing my_missingrps r:rzBooster._validate_features s9    %   T%7%7%CC))D../0     .d<1C1CDE\=1IKT, >?#\4#5#56CJ4C!ii< <<=&' Iii; ;;< SZZ(:(:MJK K/ /r9featurebins as_pandasc |j|}g}tjdj|}|D]@}tj||} |j | D cgc] } t | c} Bttj|} t| t| |n| d}tj||} tj| ddd| df} | | dddfdkD} | jdk(r|j}|j }|)t#|j%Dcgc]}d| }} |j'|}t)t*t,||}|dk(r t/d |rt4rt7| d d g S|r t4st9j:d t<| Scc} wcc}w#t.t0t2f$rd}YmwxYw)aGet split value histogram of a feature Parameters ---------- feature : The name of the feature. fmap: The name of feature map file. bin : The maximum number of bins. Number of bins equals number of unique split values n_unique, if bins == None or bins > n_unique. as_pandas : Return pd.DataFrame when pandas is installed. If False or pandas is not installed, return numpy ndarray. Returns ------- a histogram of used splitting values for the specified feature either as numpy array or pandas DataFrame. )r>z\[{0}<([\d.Ee+-]+)\]Nr)rrrcz8Split value historgam doesn't support categorical split. SplitValueCount)columnszPReturning histogram as ndarray (as_pandas == True, but pandas is not installed).)rCrqcompilerrrextendrdrJrrmaxmin histogram column_stackrrnrmrRrindexrrrErr/rKr-r.rurvrw)r rr>rrxgdumprregexprmr n_uniquenph nph_stackedrrYrXr feature_ts r:get_split_value_histogramz!Booster.get_split_value_histogram s8D)3::7CDC 63'A MMQ/Q58Q/ 0ryy()$*:3x&!Lll6-oos1vabz3q6&:; !+ad"3a"78   q ##B##Bz',T->->-@'AB'A!!g'AB !)+/S 2+>u+E C N )[<2IJ J - MME  E0C : !  !s#G G +GG('G()NNNr)r<rir>)rNT)rr) FFFFFTFrF)ubj)rFtext)r)rr)rNT)Sr4r5r6r7rrrrrr rr3rErOr rrrrrrurgrprr'rcrrrrrrrr rr$rrrr$rrr+r&rnr,r%rmrrv Objectiversr*rMetricrrrr(rrrrr#rdr"r%r)r)rtr.r5rrrHrCrNrMr.rrorrr8r9r:riris*.-1NR ;/&;/)*;/U9ic#IJK ;/  ;/zC4S>3c3h?@C uS#X# $C"8HSM2C78 sDcO# $*\l" d $$$4$ /uWeU%:;/ /b)$9:S$N#N$N'(c(i( 45Dhsm!345$ Q# Q4 Q6s6x /D6 (;*?PT,6x 5669h|&<9996x 5669h|&<999 $dHU38_5s:;}  6LPJJ*-J5=i5HJ J>A A *-A 5@A HSA A L"&" <gsl+,<< <  < <|8888S84$# %"'"&*0"D=D=D= D=  D=  D= D= D=D=(D=D= D=T+1#"&"] ] (]  ]  ]  ] ] ]  ] ~Ec2;;&6 7EDE4131916::D:B   0000 E  ((((Cc)+ ! +C$%+C$%+ +  +  +^)+ ! "C$%"" " c "J/1>#r{{*+> c5U +,, ->&JR?#r{{*+?CF? c5U +,, -?Dp@uS"++-='>p@ p@d $W $ $!L0F!L4!LL)+" DDBKK$%Dsm D  D rzz9$ % Dr9ri){r7r$rHr\rrqr]rurrabcrrcollections.abcrenumrr functoolsrinspectr r typingr r r rrrrrrrrrrrnumpyr scipy.sparser_typingrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,compatr-r.r/r0libpathr1rr3rErFr@r-rYr^rdrjrrkrrrzr~CDLLrrrrrrrrrr)rnumberrrrrrr rrIr_Pointerrr3rcr<rBrDr{r2rrrHrNr\rrrgrir8r9r:rs ## ("*ED"+:+ /S/U/ / 2uUFLL=P7Q(X{tCy&2#2%2 CDsEz):$; K%S/!:E#s(O L (4.t.u%% 1fkk1eCcM&:1 S 2 22I6;;IZ{ ;S ;T ; d38n  *3--0CGOR #Y"D"+d299o+$x.+$E8EEU{CRYYBJJ3#3&//3 * <*!** 6<< ()*'D'['T "38_59"))_OS #uS%%& &'8 )3<GKj$sj$l: : xR !8CG#445:z2%8 G  l  l  ^1 G1 hXgXv *O * bjj'*E"**bjj2H,II J 2::w'sEz):: ; |  FFr9