o ŀg@sdZddlmZddlmZmZddlZddlZddl m Z m Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZdd lmZddlmZmZddl m!m"Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)ddl*m+Z+m,Z,m-Z-ddl.m/Z/ddl0m1Z1erddl2m3Z3m4Z4ddl5m6Z6m7Z7m8Z8m9Z9ddl:m;Z;GdddZd2d+d,Z?d3d.d/Z@dS)4z] Provide user facing operators for doing the split part of the split-apply-combine paradigm. ) annotations) TYPE_CHECKINGfinalN)using_copy_on_writewarn_copy_on_write)lib)OutOfBoundsDatetime)InvalidIndexError)cache_readonly)find_stack_level) is_list_like is_scalar)CategoricalDtype) algorithms) CategoricalExtensionArray) DataFrame)ops)recode_for_groupby)CategoricalIndexIndex MultiIndex)Series) pprint_thing)HashableIterator) ArrayLikeAxisNDFrameTnpt)NDFramecseZdZUdZded<ded<ded<ded<dZd ed <fd d Zd d d ejddfd/ddZ d0d1ddZ d2d dd3dd Z e e d4d"d#Ze e d$d%Ze e d&d'Ze e d(d)Ze e d*d+Ze d5d-d.ZZS)6Groupera A Grouper allows the user to specify a groupby instruction for an object. This specification will select a column via the key parameter, or if the level and/or axis parameters are given, a level of the index of the target object. If `axis` and/or `level` are passed as keywords to both `Grouper` and `groupby`, the values passed to `Grouper` take precedence. Parameters ---------- key : str, defaults to None Groupby key, which selects the grouping column of the target. level : name/number, defaults to None The level for the target index. freq : str / frequency object, defaults to None This will groupby the specified frequency if the target selection (via key or level) is a datetime-like object. For full specification of available frequencies, please see `here `_. axis : str, int, defaults to 0 Number/name of the axis. sort : bool, default to False Whether to sort the resulting labels. closed : {'left' or 'right'} Closed end of interval. Only when `freq` parameter is passed. label : {'left' or 'right'} Interval boundary to use for labeling. Only when `freq` parameter is passed. convention : {'start', 'end', 'e', 's'} If grouper is PeriodIndex and `freq` parameter is passed. origin : Timestamp or str, default 'start_day' The timestamp on which to adjust the grouping. The timezone of origin must match the timezone of the index. If string, must be one of the following: - 'epoch': `origin` is 1970-01-01 - 'start': `origin` is the first value of the timeseries - 'start_day': `origin` is the first day at midnight of the timeseries - 'end': `origin` is the last value of the timeseries - 'end_day': `origin` is the ceiling midnight of the last day .. versionadded:: 1.3.0 offset : Timedelta or str, default is None An offset timedelta added to the origin. dropna : bool, default True If True, and if group keys contain NA values, NA values together with row/column will be dropped. If False, NA values will also be treated as the key in groups. Returns ------- Grouper or pandas.api.typing.TimeGrouper A TimeGrouper is returned if ``freq`` is not ``None``. Otherwise, a Grouper is returned. Examples -------- ``df.groupby(pd.Grouper(key="Animal"))`` is equivalent to ``df.groupby('Animal')`` >>> df = pd.DataFrame( ... { ... "Animal": ["Falcon", "Parrot", "Falcon", "Falcon", "Parrot"], ... "Speed": [100, 5, 200, 300, 15], ... } ... ) >>> df Animal Speed 0 Falcon 100 1 Parrot 5 2 Falcon 200 3 Falcon 300 4 Parrot 15 >>> df.groupby(pd.Grouper(key="Animal")).mean() Speed Animal Falcon 200.0 Parrot 10.0 Specify a resample operation on the column 'Publish date' >>> df = pd.DataFrame( ... { ... "Publish date": [ ... pd.Timestamp("2000-01-02"), ... pd.Timestamp("2000-01-02"), ... pd.Timestamp("2000-01-09"), ... pd.Timestamp("2000-01-16") ... ], ... "ID": [0, 1, 2, 3], ... "Price": [10, 20, 30, 40] ... } ... ) >>> df Publish date ID Price 0 2000-01-02 0 10 1 2000-01-02 1 20 2 2000-01-09 2 30 3 2000-01-16 3 40 >>> df.groupby(pd.Grouper(key="Publish date", freq="1W")).mean() ID Price Publish date 2000-01-02 0.5 15.0 2000-01-09 2.0 30.0 2000-01-16 3.0 40.0 If you want to adjust the start of the bins based on a fixed timestamp: >>> start, end = '2000-10-01 23:30:00', '2000-10-02 00:30:00' >>> rng = pd.date_range(start, end, freq='7min') >>> ts = pd.Series(np.arange(len(rng)) * 3, index=rng) >>> ts 2000-10-01 23:30:00 0 2000-10-01 23:37:00 3 2000-10-01 23:44:00 6 2000-10-01 23:51:00 9 2000-10-01 23:58:00 12 2000-10-02 00:05:00 15 2000-10-02 00:12:00 18 2000-10-02 00:19:00 21 2000-10-02 00:26:00 24 Freq: 7min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min')).sum() 2000-10-01 23:14:00 0 2000-10-01 23:31:00 9 2000-10-01 23:48:00 21 2000-10-02 00:05:00 54 2000-10-02 00:22:00 24 Freq: 17min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min', origin='epoch')).sum() 2000-10-01 23:18:00 0 2000-10-01 23:35:00 18 2000-10-01 23:52:00 27 2000-10-02 00:09:00 39 2000-10-02 00:26:00 24 Freq: 17min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min', origin='2000-01-01')).sum() 2000-10-01 23:24:00 3 2000-10-01 23:41:00 15 2000-10-01 23:58:00 45 2000-10-02 00:15:00 45 Freq: 17min, dtype: int64 If you want to adjust the start of the bins with an `offset` Timedelta, the two following lines are equivalent: >>> ts.groupby(pd.Grouper(freq='17min', origin='start')).sum() 2000-10-01 23:30:00 9 2000-10-01 23:47:00 21 2000-10-02 00:04:00 54 2000-10-02 00:21:00 24 Freq: 17min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min', offset='23h30min')).sum() 2000-10-01 23:30:00 9 2000-10-01 23:47:00 21 2000-10-02 00:04:00 54 2000-10-02 00:21:00 24 Freq: 17min, dtype: int64 To replace the use of the deprecated `base` argument, you can now use `offset`, in this example it is equivalent to have `base=2`: >>> ts.groupby(pd.Grouper(freq='17min', offset='2min')).sum() 2000-10-01 23:16:00 0 2000-10-01 23:33:00 9 2000-10-01 23:50:00 36 2000-10-02 00:07:00 39 2000-10-02 00:24:00 24 Freq: 17min, dtype: int64 boolsortdropna Index | None _gpr_index_grouper)keylevelfreqaxisr#r$ztuple[str, ...] _attributescs*|ddurddlm}|}t|S)Nr*r) TimeGrouper)getpandas.core.resampler-super__new__)clsargskwargsr- __class__O/var/www/html/myenv/lib/python3.10/site-packages/pandas/core/groupby/grouper.pyr1s  zGrouper.__new__NFTr+Axis | lib.NoDefaultreturnNonecCst|tur|tjurtjdttdnd}|tjurd}||_||_ ||_ ||_ ||_ ||_ d|_d|_d|_d|_d|_d|_d|_dS)Nz~Grouper axis keyword is deprecated and will be removed in a future version. To group on axis=1, use obj.T.groupby(...) instead stacklevelr)typer!r no_defaultwarningswarn FutureWarningr r(r)r*r+r#r$_grouper_deprecated_indexer_deprecated_obj_deprecatedr&binnerr'_indexer)selfr(r)r*r+r#r$r7r7r8__init__s.   zGrouper.__init__objrvalidate tuple[ops.BaseGrouper, NDFrameT]c CsF||\}}}t||jg|j|j|j||jd\}}}||_||fS)z Parameters ---------- obj : Series or DataFrame validate : bool, default True if True, validate the grouper Returns ------- a tuple of grouper, obj (possibly sorted) )r+r)r#rKr$) _set_grouper get_grouperr(r+r)r#r$rC)rHrJrK_grouperr7r7r8 _get_grouper,s  zGrouper._get_grouper) gpr_indexrR3tuple[NDFrameT, Index, npt.NDArray[np.intp] | None]c Cs|dusJ|jdur|jdurtd|jdur ||_|j|_|jdurq|j}t|dd|kr[t|tr[|jdus__name__rBr r&r[)rHr`r7r7r8rksz Grouper.axcC$tjt|jdttd|jS)Nz^.indexer is deprecated and will be removed in a future version. Use Resampler.indexer instead.r<)r@rAr>rmrBr rDrHr7r7r8rl zGrouper.indexercCrn)NzX.obj is deprecated and will be removed in a future version. Use GroupBy.indexer instead.r<)r@rAr>rmrBr rEror7r7r8rJs z Grouper.objcCrn)Nz\.grouper is deprecated and will be removed in a future version. Use GroupBy.grouper instead.r<)r@rAr>rmrBr rCror7r7r8rPrpzGrouper.groupercCs&tjt|jdttd|jjS)NzZ.groups is deprecated and will be removed in a future version. Use GroupBy.groups instead.r<)r@rAr>rmrBr rCgroupsror7r7r8rqs zGrouper.groupsstrcs8fddjD}d|}tj}|d|dS)Nc3s6|]}t|dur|dtt|VqdS)N=)r\repr).0 attr_nameror7r8 sz#Grouper.__repr__..z, ())r,joinr>rm)rH attrs_listattrscls_namer7ror8__repr__s   zGrouper.__repr__)r+r9r#r"r$r"r:r;)T)rJrrKr"r:rL)F)rJrr#r"rRr%r:rSr:rr:rr)rm __module__ __qualname____doc____annotations__r,r1rr?rIrQrMrpropertyrkrlrJrPrqr~ __classcell__r7r7r5r8r!BsN 5    (  T         r!c@s*eZdZUdZdZded<ded<ded<d ed <    dEdFddZdGddZdHdd Ze dId!d"Z e dJd$d%Z e dKd'd(Z e dLd*d+Ze dMd-d.Ze dNd0d1Ze dOd3d4Ze dOd5d6Ze dPd7d8Ze dPd9d:Ze dPd;d<Ze dPd=d>Ze dQd@dAZe dRdCdDZdS)SGroupingah Holds the grouping information for a single key Parameters ---------- index : Index grouper : obj : DataFrame or Series name : Label level : observed : bool, default False If we are a Categorical, use the observed values in_axis : if the Grouping is a column in self.obj and hence among Groupby.exclusions list dropna : bool, default True Whether to drop NA groups. uniques : Array-like, optional When specified, will be used for unique values. Enables including empty groups in the result for a BinGrouper. Must not contain duplicates. Attributes ------- indices : dict Mapping of {group -> index_list} codes : ndarray Group codes group_index : Index or None unique groups groups : dict Mapping of {group -> label_list} Nz$npt.NDArray[np.signedinteger] | None_codeszCategorical | None _all_grouperr% _orig_catsr_indexTFr`rJNDFrame | Noner#r"observedin_axisr$uniquesArrayLike | Noner:r;c Cs||_||_t||} d|_d|_||_||_||_||_||_ ||_ | |_ |j } | durIt |tr8|| } n|} | durA| } nx| } | | } npt | trz|jdusUJ| j|jdd\}}||_t |tjrk|} nN|jdj}t||jjd} n?t | ttttjfst| dddkrtt | }t!d|d|| } t"| d rt#| t#|kst$| }d |}t%|t | tjr| j&j'd vrt| (} nt t| d dt)r| j*|_t+| ||\} |_| |_dS) NFrKrrUndim Grouper for '' not 1-dimensional__len__z9Grouper result violates len(labels) == len(data) result: mMdtype),r) _orig_grouper_convert_grouperrrr_sortrJ _observedr_dropna_uniques_ilevelr]rget_level_valuesmapr!rQr BinGrouper groupingsgrouping_vectorr result_indexrTrrnpndarrayr\rrr>r[hasattrlenrAssertionErrorrrXto_numpyr categoriesr)rHr`rPrJr)r#rrr$rrilevel index_levelmapper newgroupernewobjngtgrpererrmsgr7r7r8rIsn              zGrouping.__init__rrcCsd|jdS)Nz Grouping(ryrUror7r7r8r~xszGrouping.__repr__rcC t|jSN)iterindicesror7r7r8__iter__{s zGrouping.__iter__cCst|jdd}t|tS)Nr)r\rr]r)rHrr7r7r8_passed_categorical~s zGrouping._passed_categoricalrcCsb|j}|dur |jj|St|jttfr|jjSt|jt j r%|jj jSt|jtr/|jjSdSr) rrrfr]rrrrTrr BaseGrouperr)rHrr7r7r8rTs   z Grouping.name int | NonecCsL|j}|dur dSt|ts$|j}||jvrtd|d|j|S|S)zS If necessary, converted index level name to index level position. NzLevel z not in index)r)r]intrrfrr`)rHr)r`r7r7r8rs   zGrouping._ilevelrcCrr)r _group_indexror7r7r8ngroups zGrouping.ngroups$dict[Hashable, npt.NDArray[np.intp]]cCs(t|jtjr |jjSt|j}|Sr)r]rrrrr_reverse_indexer)rHvaluesr7r7r8rs zGrouping.indicesnpt.NDArray[np.signedinteger]cCs |jdS)Nr)_codes_and_uniquesror7r7r8codesrzGrouping.codesrcCs*|jdur |jjS|jr|jjS|jdS)v Analogous to result_index, but holding an ArrayLike to ensure we can retain ExtensionDtypes. Nr)r _result_index_valuesrrrror7r7r8_group_arraylikes  zGrouping._group_arraylikecCtjdttd|jS)rzOgroup_arraylike is deprecated and will be removed in a future version of pandascategoryr=)r@rArBr rror7r7r8group_arraylikes zGrouping.group_arraylikecCs4|jdur|j}t|tsJ|j}||S|jSr)rrr]rrset_categories)rH group_idxcatsr7r7r8rs  zGrouping._result_indexcCr)NzLresult_index is deprecated and will be removed in a future version of pandasr)r@rArBr rror7r7r8r zGrouping.result_indexcCs|j\}}|js_|jr_t|tsJ|jr-|t|kr-tjt |j dg|j dd}n2t|dkr_|j }|j dk}|j |dkr_t|j d|}t |j |d}tj||j dd}tj||jdS)NFrrrU)rrrr]rrrany from_codesrappendrrrargmaxr nunique_intsinsertr _with_inferrT)rHrrcatna_idx na_unique_idx new_codesr7r7r8rs"   zGrouping._group_indexcCr)NzKgroup_index is deprecated and will be removed in a future version of pandasr)r@rArBr rror7r7r8 group_indexrzGrouping.group_index/tuple[npt.NDArray[np.signedinteger], ArrayLike]c Csp|jr||j}|j}|jr!t|j}||dk}|jr t |}nt t |}t j |||jdd}|j}|jso|dk}t|ro|jrPt |}t|||}n|}t|d|}t||k|d|}t|||}|jsx||j}||fSt|jtjr|jj}|jjj}||fS|jdurt |j|jd}|j}|j}||fStj|j|j|jd\}}||fS)NrF)rrorderedrKrr)r)r#use_na_sentinel)rrrrrunique1drrrr#arangerrrrrrwhererrreorder_categoriesrr]rr codes_inforrr factorize) rHrrucodesrrna_maskna_coderr7r7r8r sP          zGrouping._codes_and_uniquesdict[Hashable, np.ndarray]cCs tj|j|jdd}|j|S)NFr)rrrrrgroupby)rHrr7r7r8rqHs zGrouping.groups)NNNTFFTN)r`rrJrr#r"rr"rr"r$r"rrr:r;r)r:rr:r")r:r)r:r)r:r)r:r)r:r)r:rr)r:r)r:r)rmrrrrrrIr~rr rrTrrrrrrrrrrrrrqr7r7r7r8rsZ   i              >rTFrJrr+rr#r"rrKr$r:5tuple[ops.BaseGrouper, frozenset[Hashable], NDFrameT]c s|}|durtt|tr,t|rt|dkr|d}|dur+t|r+||}d}nHt|rIt|} | dkr=|d}n | dkrEtdtdt|trd|j |krctd|d |n |dksl|dkrptd d}|}t|t r|j d d \} |j dur| tfS| t|j hfSt|tjr|tfSt|ts|g} d } n |} t| t|k} td d | D} tdd | D}tdd | D}| s|s|s| r|durttrtfdd | D}nttsJtfdd | D}|st| g} t|ttfr&|dur#dgt|} |}n|gt| }g}t}d fdd }d fdd }t| |D]\}}||rWd}||j nd||rjdkr|vr|rqj||dd||}}}|jdkrtd|d||n,j||drd |d}}}nt|t|t r|j dur||j d}nd }t|t st |||||||dn|}|!|qEt|dkrtrtdt|dkr|!t t"gddt#j$gt#j%dtj||||d} | t|fS)!a Create and return a BaseGrouper, which is an internal mapping of how to create the grouper indexers. This may be composed of multiple Grouping objects, indicating multiple groupers Groupers are ultimately index mappings. They can originate as: index mappings, keys to columns, functions, or Groupers Groupers enable local references to axis,level,sort, while the passed in axis, level, and sort are 'global'. This routine tries to figure out what the passing in references are and then creates a Grouping for each one, combined into a BaseGrouper. If observed & we have a categorical grouper, only show the observed values. If validate, then check for key/level overlaps. NrrzNo group keys passed!z*multiple levels only valid with MultiIndexz level name z is not the name of the rz2level > 0 or level < -1 only valid with MultiIndexFrcss"|] }t|p t|tVqdSr)callabler]dictrugr7r7r8rws zget_grouper..css|] }t|ttfVqdSr)r]r!rrr7r7r8rwscss&|]}t|tttttjfVqdSr)r]listtuplerrrrrr7r7r8rw c3s&|]}|jvp|jjvVqdSr)columnsr`rfrrJr7r8rwrc3s|] }|jjvVqdSr)r`rfrrr7r8rwsr:r"c sPt|s&jdkr dSjd}z||WdStttfy%YdSwdS)NrFrT)_is_label_likeraxesget_locrb TypeErrorr )r(itemsrr7r8 is_in_axiss   zget_grouper..is_in_axisc st|dsdSts tr7z|j}Wnttttfy"YdSwt|t r5t|t r5|j |j dSdSz||juWSttttfyMYdSw)NrTFr) rrrrTrb IndexErrorr rr]r_mgrreferences_same_values)gprobj_gpr_columnrr7r8 is_in_objs$  zget_grouper..is_in_objTrZrr)rJr)r#rrr$r)r)r#r$r)&rcr]rr rr rr[rrrT_get_axis_namer!rQr( frozensetrrrrrallrcomasarray_tuplesafersetzipaddr_check_label_or_level_ambiguity_is_level_referencerbrrrrrhintp)rJr(r+r)r#rrKr$ group_axisnlevelsrPkeysmatch_axis_length any_callable any_groupers any_arraylikeall_in_columns_indexlevelsr exclusionsrrrrrTpingr7rr8rNNs                       $rNcCst|ttfp|duot|Sr)r]rrrr )valr7r7r8r8srrcCst|tr|jSt|tr|j|r|jS||jSt|tr$|jSt|t t t t t jfrIt|t|kr;tdt|t t frGt|}|S|S)Nz$Grouper and axis must be same length)r]rr.rr`equalsrreindexrrrrrrrrr[r r )r+rPr7r7r8r<s      r)NrNTFTT)rJrr+rr#r"rr"rKr"r$r"r:rr)r+r)Ar __future__rtypingrrr@numpyrpandas._configrr pandas._libsrpandas._libs.tslibsr pandas.errorsr pandas.util._decoratorsr pandas.util._exceptionsr pandas.core.dtypes.commonr r pandas.core.dtypes.dtypesr pandas.corerpandas.core.arraysrrpandas.core.commoncorecommonr pandas.core.framerpandas.core.groupbyrpandas.core.groupby.categoricalrpandas.core.indexes.apirrrpandas.core.seriesrpandas.io.formats.printingrcollections.abcrrpandas._typingrrrrpandas.core.genericr r!rrNrrr7r7r7r8sZ              )i  k