logo
down
shadow

Pandas DataFrame, computing the Time Difference between one row and other row which satisfies a condition


Pandas DataFrame, computing the Time Difference between one row and other row which satisfies a condition

By : user3099399
Date : January 11 2021, 03:32 PM
hope this fix your issue computing the Time Difference based on Two rows, picked upon a condition. First Row is the Current Row, and Second Row is picked upon a condition. Condition can be Value==1 for example. , Load a test DataFrame:
code :
df = pd.DataFrame({'year': [2019] * 5,
                   'month': [8] * 5,
                   'day': [16] * 5,
                   'hour': [12, 12, 12, 12, 13],
                   'minute': [1, 2, 3, 4, 5]})

df = pd.DataFrame(pd.to_datetime(df), columns=['Time_Stamp'])
df['Event_Master'] = [0, 0, 1, 0, 1]
           Time_Stamp  Event_Master
0 2019-08-16 12:01:00             0
1 2019-08-16 12:02:00             0
2 2019-08-16 12:03:00             1
3 2019-08-16 12:04:00             0
4 2019-08-16 13:05:00             1
df['Next_Timestamp'] = df[df.Event_Master == 1].Time_Stamp
df['Next_Timestamp'].fillna(method='backfill', inplace=True)

df['TimeDiff'] = df.Next_Timestamp - df.Time_Stamp
           Time_Stamp  Event_Master      Next_Timestamp TimeDiff
0 2019-08-16 12:01:00             0 2019-08-16 12:03:00 00:02:00
1 2019-08-16 12:02:00             0 2019-08-16 12:03:00 00:01:00
2 2019-08-16 12:03:00             1 2019-08-16 12:03:00 00:00:00
3 2019-08-16 12:04:00             0 2019-08-16 13:05:00 01:01:00
4 2019-08-16 13:05:00             1 2019-08-16 13:05:00 00:00:00
df.drop(['Next_Timestamp'], axis=1, inplace=True)
           Time_Stamp  Event_Master TimeDiff
0 2019-08-16 12:01:00             0 00:02:00
1 2019-08-16 12:02:00             0 00:01:00
2 2019-08-16 12:03:00             1 00:00:00
3 2019-08-16 12:04:00             0 01:01:00
4 2019-08-16 13:05:00             1 00:00:00


Share : facebook icon twitter icon
Computing percentage difference between pandas dataframe rows

Computing percentage difference between pandas dataframe rows


By : vinhnguyen1211
Date : March 29 2020, 07:55 AM
will be helpful for those in need You can use DataFrameGroupBy.pct_change with groupby by column region:
code :
df['new'] = df.groupby('region')['val'].pct_change()
print (df)
   region    year       val       new
0     1.0  2015.0  677545.7       NaN
1     1.0  2016.0  681976.1  0.006539
2     1.0  2017.0  686406.5  0.006496
3     2.0  2015.0  617545.7       NaN
4     2.0  2016.0  641976.1  0.039560
5     3.0  2017.0  656406.5       NaN
Python Pandas: test if any column of a DataFrame satisfies a condition

Python Pandas: test if any column of a DataFrame satisfies a condition


By : Stacey hunter
Date : March 29 2020, 07:55 AM
should help you out ne is the method form of !=. I use that so that pipelining any looks nicer. I use any(axis=1) to find if any are true in a row.
code :
df['indicator'] = df[columns].ne(0).any(axis=1)
Num of occurances where Column difference in DataFrame satisfies a condition

Num of occurances where Column difference in DataFrame satisfies a condition


By : naivegeek
Date : March 29 2020, 07:55 AM
Hope this helps I think need compare by eq for == with last 6 values by tail and count values by sum:
code :
out = mydf['Close'].tail(6).eq(mydf['Open'].tail(6)).sum()
out = mydf['Close'].tail(6).sub(mydf['Open'].tail(6)).abs().eq(0).sum()
Create a new Dataframe based on Time Difference and a condition on columns in pandas dataframe

Create a new Dataframe based on Time Difference and a condition on columns in pandas dataframe


By : user2960369
Date : March 29 2020, 07:55 AM
wish help you to fix your issue I would first use a groupby on ticketID to compute a rank per ticket, then pivot the dataframe using that rank as column and ticketID as index to obtain the expected data.
After sorting the columns you get the expected dataframe. Time to rename the columns and reset the index to have a nice dataframe. Code could be:
code :
df['rank'] = df.groupby('ticketID').apply(lambda x:
                                          pd.Series(range(len(x)))).values
resul = df.pivot('ticketID', 'rank').fillna('')
resul.columns = resul.columns.swaplevel()
resul.sort_index(axis=1,inplace=True, level=0, sort_remaining=False)
resul.columns = ['{1}_{0}'.format(*c) for c in resul.columns]
resul.reset_index(inplace=True)
   ticketID             ChangeDate_0 OldStatus_0                  NewStatus_0             ChangeDate_1                  OldStatus_1                        NewStatus_1             ChangeDate_2                        OldStatus_2     NewStatus_2
0   1012327  2019-03-18 09:00:32.903      R or O   Action mail sent to client  2019-03-18 09:21:34.820   Action mail sent to client                Response Client - R  2019-03-18 09:34:21.890                Response Client - R  Status Updated
1   1012328  2019-03-18 07:00:09.960      R or O         ticket Closed - None  2019-03-18 07:09:31.420         ticket Closed - None                     Status Updated                                                                            
2   1012329  2019-03-18 06:52:03.490      R or O    ticket Closed - Satisfied  2019-03-18 07:09:33.433    ticket Closed - Satisfied                     Status Updated                                                                            
3   1012330  2019-03-18 10:25:13.493      R or O  Action mail sent to Service  2019-03-18 10:55:20.963  Action mail sent to Service  ticket Closed - Service Responded  2019-03-18 11:02:05.327  ticket Closed - Service Responded  Status Updated
4   1012332  2019-03-18 09:00:41.967      R or O   Action mail sent to client  2019-03-18 10:24:20.150   Action mail sent to client                Response Client - R  2019-03-18 10:32:40.717                Response Client - R  Status Updated
error in computing difference between two date time stamp [yyyy-mm-dd hh:mm:ss.000] columns in seconds in pandas datafra

error in computing difference between two date time stamp [yyyy-mm-dd hh:mm:ss.000] columns in seconds in pandas datafra


By : user3625422
Date : March 29 2020, 07:55 AM
Hope this helps I have a dataframe which has two columns 'Etime' and 'Stime' which contains end and start timestamp and sample of which is shown below: , Try this:
code :
date_format = '%Y-%m-%d %H:%M:%S.%f'
df['Duration'] = [(datetime.strptime(df.loc[x,'Etime'], date_format ) -
                     datetime.strptime(df.loc[x,'Stime'], date_format)).seconds
                         for x in df.index]
Related Posts Related Posts :
  • Tuning the hyperparameter with gridsearch results in overfitting
  • some coordinates that I extracted from geocoder in Python are not saving in the variable I created
  • 7C in cs circles- python Im not sure what is wrong with this yet
  • How to fix 'AttributeError: 'list' object has no attribute 'shape'' error in python with Tensorflow / Keras when loading
  • python - thread`s target is a method of an object
  • Retrieve Variable From Class
  • What is the reason for matplotlib for printing labels multiple times?
  • Why would people use ThreadPoolExecutor instead of direct function call?
  • When clear_widgets is called, it doesnt remove screens in ScreenManager
  • Python can't import function
  • Pieces doesn't stack after one loop on my connect4
  • How to change font size of all .docx document with python-docx
  • How to store a word with # in .cfg file
  • How to append dictionaries to a dictionary?
  • How can I scrape text within paragraph tag with some other tags then within the paragraph text?
  • Custom entity ruler with SpaCy did not return a match
  • Logging with two handlers - one to file and one to stderr
  • How to do pivot_table in dask with aggfunc 'min'?
  • This for loop displays only the last entry of the student record
  • How to split a string by a specific pattern in number of characters?
  • Python 3: how to scrape research results from a website using CSFR?
  • Setting the scoring parameter of RandomizedSeachCV to r2
  • How to send alert or message from view.py to template?
  • How to add qml ScatterSeries to existing qml defined ChartView?
  • Django + tox: Apps aren't loaded yet
  • My css and images arent showing in django
  • Probability mass function sum 2 dice roll?
  • Cannot call ubuntu 'ulimit' from python subprocess without using shell option
  • Dataframe Timestamp Filter for new/repeating value
  • Problem with clicking select2 dropdownlist in selenium
  • pandas dataframe masks to write values into new column
  • How to click on item in navigation bar on top of page using selenium python?
  • Add multiple EntityRuler with spaCy (ValueError: 'entity_ruler' already exists in pipeline)
  • error when replacing missing ')' using negative look ahead regex in python
  • Is there a way to remove specific strings from indexes using a for loop?
  • select multiple tags by position in beautifulSoup
  • pytest: getting AttributeError: 'CaptureFixture' object has no attribute 'readouterror' capturing stdout
  • Shipping PyGObject/GTK+ app on Windows with MingW
  • Python script to deduplicate lines in multiple files
  • How to prevent window and widgets in a pyqt5 application from changing size when the visibility of one widget is altered
  • How to draw stacked bar plot from df.groupby('feature')['label'].value_counts()
  • Python subprocess doesn't work without sleep
  • How can I adjust 'the time' in python with module Re
  • Join original np array with resulting np array in a form of dictionary? multidimensional array? etc?
  • Forcing labels on histograms in each individual graph in a figure
  • For an infinite dataset, is the data used in each epoch the same?
  • Is there a more efficent way to extend a string?
  • Is it possible to do this loop in a one-liner?
  • invalid literal for int() with base 10: - django
  • Why does my code print a value that I have not assigned as yet?
  • the collatz func in automate boring stuff with python
  • How to find all possible combinations of parameters and funtions
  • about backpropagation deep neural network in tensorflow
  • Sort strings in pandas
  • How do access my flask app hosted in docker?
  • Replace the sentence include some text with Python regex
  • Counting the most common element in a 2D List in Python
  • logout a user from the system using a function in python
  • mp4 metadata not found but exists
  • Django: QuerySet with ExpressionWrapper
  • shadow
    Privacy Policy - Terms - Contact Us © festivalmusicasacra.org