chris.mclennon
3 questions
1
votes
1
answer
728
views
How do I select an ambiguous column reference? [duplicate]
This question already has an answer here:
Enable case sensitivity for spark.sql globally
1 answer
Here's some sample code illustrating what I'm trying to do. There is a dataframe with columns companyid and companyId. I want to select companyId, but the reference is ambiguous. How do I unambiguously...
1
votes
0
answer
251
views
How to import referenced files in ETL scripts?
I have a script which I'd like to pass a configuration file into. On the Glue jobs page, I see that there is a 'Referenced files path' which points to my configuration file. How do I then use that file within my ETL script?
I've tried from configuration import *, where the referenced file name is co...
1
votes
2
answer
46
views
How can I run a function over a rolling window on each collective key in Spark?
I am handling some event data, and I'd like to run a function that loops down an ordered list of values for a given key and return a value. I'm not sure how to do this, or if it's possible using Spark, and am hoping for a tip in the right direction.
Specifically, I have some event data, and I'd like...