chrisaycock

12

votes
2

answer
5.8k

views

Unable to query a local variable in pandas 0.14.0

I can query an explicit value: fills.query('Symbol=='BUD US'') Now I want to query a variable: In [40]: my_symbol Out[40]: 'BUD US' In pandas 0.13.1, I could simply use: fills.query('Symbol==my_symbol') This is no longer permitted in pandas 0.14.0 because local variables must be referred to explicit...
chrisaycock
1

votes
1

answer
2.5k

views

How do I convert a timestamp to seconds-since-midnight?

Suppose I have this timestamp: In [146]: t Out[146]: datetime.time(11, 2, 5, 110521) How do I convert this to the the number of seconds since midnight? One approach is to first convert this to a timedelta: In [147]: datetime.timedelta(hours=t.hour, minutes=t.minute, seconds=t.second, microseconds=t....
chrisaycock
1

votes
1

answer
322

views

Why does pandas give NaN for column values when aggregating a column that doesn't exist?

Let's I want the sum of numbers by letter in the below DataFrame: In [10]: df Out[10]: letter number 0 A 1 1 A 2 2 B 3 3 B 4 4 C 5 5 C 6 [6 rows x 2 columns] This is really easy to accomplish: In [11]: df.groupby('letter')[['number']...
chrisaycock
2

votes
2

answer
293

views

Using 3rd party library's callbacks (no file descriptor) in an event-driven / asynchronous framework

The goal is to have a single-threaded event-driven framework. The end-user will do something like: class MyEventFramework : public EventFramework { protected: virtual void onData (const Data& data); virtual void onReport (const Report& report); virtual void onUserRequest (const UserRequest& userRequ...
chrisaycock
5

votes
1

answer
259

views

Performance issues with groupby's last in pandas

I have a DataFrame with two columns and a little over one-hundred thousand elements. In [43]: df.head(10) Out[43]: localtime ref 4 2014-04-02 12:00:00.273537 139058754703810577 5 2014-04-02 12:00:02.223501 139058754703810576 6 2014-04-02 12:00:03.518817 139058754703810576 7 20...
chrisaycock
2

votes
2

answer
2.7k

views

Reading key-value pairs into Pandas

Pandas makes it really easy to read a CSV file: pd.read_table('data.txt', sep=',') Does Pandas having something similar for a file with key-value pairs? I came-up with this: pd.DataFrame([dict([p.split('=') for p in l.split(',')]) for l in open('data.txt')]) If not built-in, then perhaps something m...
chrisaycock
2

votes
1

answer
595

views

Querying a single-row DataFrame with AND'ed conditionals

Using pandas 0.13, I have a single-row DataFrame: In [23]: is_same Out[23]: bid bidsize ask asksize 168 False False False False [1 rows x 4 columns] I can query this with Python syntax: In [24]: is_same.bid & is_same.ask Out[24]: 168 False dtype: bool The trouble comes when I use the...
chrisaycock
2

votes
1

answer
1.1k

views

How to decrease size overhead of HDFStore?

I am experimenting with different pandas-friendly storage schemes for tick data. The fastest (in terms of reading and writing) so far has been using an HDFStore with blosc compression and the 'fixed' format. store = pd.HDFStore(path, complevel=9, complib='blosc') store.put(symbol, df) store.close()...
chrisaycock
13

votes
1

answer
3.5k

views

Using Clang's scan-build with scons and C++11

I've added everything to my $PATH and I've tweaked my SConstruct to set the appropriate environment variables, as per these answers [ 1, 2, 3 ]. Now when I run scan-build --use-c++=`which clang++` scons the build begins, and I can see the process forked by scons is /path/to/c++-analyzer ... -std=c+...
chrisaycock
43

votes
4

answer
44.3k

views

Shift elements in a numpy array

Following-up from this question years ago, is there a canonical 'shift' function in numpy? I don't see anything from the documentation. Here's a simple version of what I'm looking for: def shift(xs, n): if n >= 0: return np.r_[np.full(n, np.nan), xs[:-n]] else: return np.r_[xs[-n:], np.full(-n, np.n...
chrisaycock
4

votes
2

answer
77

views

How does CPython's interpreter know to print the result of the last expression?

I've been digging around the source code to figure-out at which point the result is printed. For example: >>> x = 1 >>> x + 2 3 The above two statements are compiled to: 1 0 LOAD_CONST 0 (1) 3 STORE_NAME 0 (x) 6 LOAD_CONST 1 (None) 9 RETURN_VALUE a...
chrisaycock
69

votes
6

answer
14.6k

views

How do I delete a remote branch in Git? [duplicate]

This question already has an answer here: How do I delete a Git branch both locally and remotely? 39 answers I created a branch notmaster to commit as well as push some changes. When I was finished with that branch, I merged the changes back into master, pushed them out, and then deleted the local...
chrisaycock
77

votes
2

answer
56.3k

views

How do I revert to a previous package in Anaconda?

If I do conda info pandas I can see all of the packages available. I updated my pandas to the latest this morning, but I need to revert to a prior version now. I tried conda update pandas 0.13.1 but that didn't work. How do I specify which version to use?
chrisaycock
4

votes
1

answer
651

views

Error reporting in Boost Spirit

I have the following error handler at the bottom of my parser grammar: qi::on_error( launch, std::cerr
chrisaycock
2

votes
2

answer
125

views

How do I convert varargs to a single list even if the parameter is already a list?

I want a function that returns varargs as a list, such that ensure_list(1, 2, 3) returns [1,2,3] But the caveat is that I want ensure_list([1, 2, 3]) to return the same value. I tried def ensure_list(Object... args) { if (args instanceof List) { return args } else { return Arrays.asList(args) } } b...
chrisaycock
3

votes
2

answer
426

views

Adapting an empty struct for Boost Fusion

I have an empty struct: struct MyStruct {}; I want to adapt this to be used by Boost Fusion. Ordinarily I would use BOOST_FUSION_ADAPT_STRUCT, but this macro requires two parameters to name the struct and list its fields. Is there anyway to tell the macro that the fields are non-existent? Motivation...
chrisaycock
2

votes
1

answer
60

views

Pandas DataFrame fails on index but Series succeeds

Given US market hours: In [220]: market_hours = pandas.date_range(date + ' 09:30:00', date + ' 16:00:00', freq='15min', tz='US/Eastern').tz_convert('UTC') In [221]: market_hours Out[221]: [2014-04-29 13:30:00+00:00, ..., 2014-04-29 20:00:00+00:00] Length: 27, Freq: 15T, Timezone: UTC I can resample(...
chrisaycock
8

votes
3

answer
6k

views

Using commas inside a macro without parenthesis: How can I mix and match with a template?

Consider a simple macro: #define ECHO(x) x ECHO(foo(1, 2)) This produces the exact output we expect: foo(1, 2) The above example works because the parenthesis adjacent to the function call are recognized by the preprocessor. Now consider what happens if I use a template instead of a function call: E...
chrisaycock
2

votes
1

answer
668

views

Why would std::unordered_map::emplace() fail?

I have a std::unordered_map that I emplace() an object to via: my_map.emplace(std::piecewise_construct, std::forward_as_tuple(key), std::forward_as_tuple(value1, value2)); This fails at some point during runtime with a false in the second position of the returned tuple. Is there a way to get more in...
chrisaycock
2

votes
2

answer
346

views

Unable to subtract datetime64 using DataFrame.eval()

Given a DataFrame with a couple of timestamps: In [88]: df.dtypes Out[88]: Time datetime64[ns] uniqstime datetime64[ns] dtype: object If I call eval(), I get a type error: In [91]: df.eval('since = Time - uniqstime') ... ValueError: unkown type timedelta64[ns] (By the way, 'unknow...
chrisaycock