kee

1

votes
1

answer
496

views

How to access GET content from SimpleHttpOperator

I understand that by setting xcom_push=True in SimpleHttpOperator I can access the returned data from Xcom from How to access the response from Airflow SimpleHttpOperator GET request. But it is not very clear to me how I can do that. Is that by creating a PythonOperator with a callback and calling x...
kee
1

votes
2

answer
1.4k

views

BigQuery: Syntax error: Unexpected keyword LEFT

I got this error of 'Syntax error: Unexpected keyword LEFT' from the following SQL (standard SQL) in BigQuery: select left(cast(ts as string), 16) from temp.loc limit 1; 'ts' is a timestamp field and I wanted to get upto minutes of timestamp. Any idea?
kee
1

votes
2

answer
1.3k

views

Python regex match square bracket issue

I am trying to match a datatime inside square brackets and I thought prefixing '\' would be the way to encode square brackets but somehow it didn't work. Here is my code: import re line_nginx = re.compile(r'''\[(?P\S+) -700\]''', re.IGNORECASE) match = line_nginx.match('[07/Oct/2014:19:43:08 -0700]...
kee
1

votes
1

answer
22

views

Merging records by 1+ common elements

I have a hive table with the following schema: int key1 # this is unique array key2_list Now I want to merge records here if their key2_lists have any common element(s). For example, if record A has (10, [1,3,5]) and B has (12, [1,2,4]), I want to merge it as ([10,12], [1,2,3,4,5]) or ([1,2,3,4,5])....
kee
1

votes
5

answer
112

views

SQL count condition with 12months table

I have the following table Named: LISTON | TYPE | MONTHS | New | Old | +------+--------+-----+-----+ | A | FEB | Y | N | | A | MAY | Y | N | | A | MAY | N | Y | | B | MAY | Y | N | | A | MAY | Y | N | | C | MAY | Y | N | | D | MAY |...
Ping Kee Ng
1

votes
1

answer
296

views

Superset: How to group by month off a timestamp field from Redshift

I am trying to show some trend over month in Superset from a table which has a timestamp field called created_at but have no idea how to get it right. The SQL query generated from this is the followings: SELECT DATE_TRUNC('month', created_at) AT TIME ZONE 'UTC' AS __timestamp, SUM(cost) AS 'SUM(cost...
kee
1

votes
0

answer
37

views

BigQuery: schema autodetection of JSON couldn't recognize a field appearing later in the JSON input file

I found that BigQuery's schema autodetection doesn't recognize a field if that doesn't appear in the beginning of an input JSON file. I have this field named 'details' which is a record type. In the first 2K rows of the JSON input file, this field doesn't have any sub-fields. But then in 2,698 rows...
kee
1

votes
1

answer
70

views

Returning array in C, Sudoku Solver

So I'm creating a sudoku solver in C. Here's my full code as of now, I've mostly been using python and just got into C, I basically converted a lot of python functions to C to get this but I think it'll work: #include #include int is_empty(); int possible_v(); int solver(); int main(){ int s_array...
Yee Kee
1

votes
0

answer
88

views

GCP: Where to schedule PubSub subscriber which writes to BigQuery

I need to write to BigQuery from PubSub in Python. I tested some async subscriber code and it works fine. But this needs to run continuously and I am not 100% sure where to schedule this. I have been using Cloud Composer (Airflow) but it doesn't look like an ideal fit and it looks like Dataflow is t...
kee
0

votes
0

answer
9

views

display value from an array to view in Jquery

I want to add math, eng and lit point plus one more if these points less than 10(just condition from my exercise). HTML STT MSSV NAME MATH ENGLISH LITERATURE GRADE FUNCTION Jquery var dssv = []; $('#plusPoint').click(() => { $('tr').each((index, item) => { if (index > 0) { var mssv = $('#mssv').val(...
KEE
1

votes
0

answer
13

views

Python - Defining output of query to variables

I am trying to call the output of SQL query to variables as below: cur = conn.cursor() cur.execute('''select store_name,DATE_TRUNC('month',bill_date)as month,count(*) from sales''') monthly_report = dwh_cur.fetchall() The output of this file is as below: store_name,month,count store_a,jan,100 store_...
hello kee
1

votes
1

answer
40

views

Pandas - Comparing two Dataframe and finding difference [duplicate]

This question already has an answer here: How to implement 'in' and 'not in' for Pandas dataframe 6 answers I have two Dataframes with some sales data as below: df1: prod_id,sale_date,new 101,2019-01-01,101_2019-01-01 101,2019-01-02,101_2019-01-02 101,2019-01-03,101_2019-01-03 101,2019-01-04,101_20...
hello kee
1

votes
1

answer
44

views

Validation and button problem before next page

I am creating a multi form. When I do not input any words and click the next button, the system has a display function validation but it also goes to next page automaticly. So, what I need todo, is detect when I input a wrong format or empty string, so the system will make a validation and not auto...
yun kee
1

votes
3

answer
82

views

Filling and Printing a 2D array

So I have a 2D array that I want to use later. Right now I just want to fill the empty spots. So far I've just been messing around with array types and different default values. From my understanding a new array is filled with '0', I have tried NULL aswell. int r = 5; int c = 5; int i; int j; int k...
Yee Kee
1

votes
1

answer
38

views

Peptide monoisotopic calculating using python

I'm making Calculator for Peptide monoisotopic and have some problems that I cannot solve. import re aminoacid = { 'I': 'C6H13NO2', 'L': 'C6H13NO2', 'K': 'C6H14N2O2', 'M': 'C5H11NO2S', 'F': 'C9H11NO2', 'T': 'C4H9NO3', 'W': 'C11H12N2O2', 'V': 'C5H11NO2', 'R': 'C6H14N4O2', 'H': 'C6H9N3O2', 'A': 'C3H7N...
Yang Kee Won
1

votes
1

answer
30

views

Find the student has highest point in JQUERY?

I used var average_point = '' + sv[2] + '' + '' + sv[3] + '' +'' + sv[4] + ''; I want to take value from single element in HTML. But this code was not correct. How can I take the value from HTML? HTML Jquery These codes to add the element in an array, and display via from HTML(above). I use HTML...
KEE
1

votes
1

answer
575

views

Hadoop On Demand [closed]

Is this Apache project (Hadoop On Demand) still actively being developed? I couldn't find any recent documentation but the concept (or motivation of the project) seems to be quite intriguing. My use case is to build relative large cluster (in-house) and allocate/deallocate portion of it to different...
kee
1

votes
1

answer
2.7k

views

PIG script execution through java and grunt

Somehow if I use grunt shell to execute a pig script, it works fine but if I try java mode, it shows 'Failed to create DataStorage' error somehow. Grunt mode command is 'pig -x mapreduce test1.pig' Java mode command is 'java -cp $PIGDIR/pig-0.9.2.jar:$HADOOP_CONF_DIR test1.pig' I am wondering what'...
kee
1

votes
1

answer
817

views

How can I check the size of input (in bytes) for a mapper?

I noticed among 500 mappers there are almost 3x completion time difference. When I checked the logs (thru JobTracker web interface), I found that the difference is mainly in the 1st spill timing ('Finished spill 0'). This seems to imply that the input file size difference per mapper isn't really a...
kee
1

votes
1

answer
313

views

ElasticMapReduce: Is it possible to reuse already allocated EMR cluster?

I specified --alive option in EMR CLI when I created a new cluster and I am wondering if it is possible to reuse the cluster in launching another job? I can't find any relevant option to get some kind of ID for the cluster? So does that mean that it is not possible to do so?
kee
1

votes
1

answer
378

views

MapReduce FIFO scheduler

I have a question about MapReduce FIFO scheduler. I understand jobs are executed as they arrive in the queue (as long as they have the same priority). My understanding is that the next job will wait until current job is done. But what if current job is not completely consuming the capacity and the n...
kee
1

votes
2

answer
193

views

jQuery slider trouble

Hi I'm creating a simple jQuery slide show. The requirement for the slide is simple, just the slide just flip continuously towards the left. Most of the tutorial shown on line is pretty advanced and fancy. But then I came across one that is pretty much almost what I want. http://www.webchiefdesign.c...
Weng Kee Chang
1

votes
1

answer
201

views

JS plugins in template of AngularJS app don't work

I'm trying to run a JS plugin in an Angular app. So I have two files: index.html and view.html (the template). The latter is the partial loaded in the ng-app DIV inside index.html. Now, jQuery and the plugin are declared in index.html. Since the implementation in view.html (the template) is, technic...
Kee
1

votes
1

answer
160

views

Changes in TokenizerFactory class in SOLR/Lucene 4.5.1

Now TokenizerFactory doesn't have setLuceneMatchVersion and init method and also expects Map from its constructor. I have been creating TokenizerFactory instance using newInstance method of ResourceLoader and then pass Map parameter thru init method like the following: tokenizerFactory = (TokenizerF...
kee
1

votes
5

answer
683

views

Java String.matches regex

I am trying to see if a given host name appears in a list of hosts in the form of comma separated string like the following: String list = 'aa.com,bb.com,cc.com,dd.net,ee.com,ff.net'; String host1 = 'aa.com'; // should be a match String host2 = 'a.com'; // shouldn't be a match String host3 = 'ff.ne...
kee
1

votes
1

answer
307

views

Google Maps - Heatmap implementation based on?

There are different kinds of heatmaps: https://gis.stackexchange.com/questions/256/how-to-build-effective-heat-maps Just a quick question, is the Google Maps's implementation of heat maps based on Concentration of points (e.g., kernel density) or Distributions of attribute values? Or is this true?...
Kee Kiat Koo
1

votes
1

answer
193

views

Datastax Opscenter issue: dashboard timeout

I installed Datastax community version in an EC2 server and it worked fine. After that I tried to add one more server and I see two nodes in the Nodes menu but in the main dashboard I see the following error: Error: Call to /Test_Cluster__No_AMI_Parameters/rc/dashboard_presets/ timed out. One potent...
kee
1

votes
1

answer
67

views

Tajo: Does tsql need Hadoop?

I know Tajo requires Hadoop to be installed first. But I am not very sure bin/tsql. Is Hadoop required for tsql to run? If so, is there any plan to make it lighter? Any insight/help would be appreciated.
kee
1

votes
1

answer
31

views

Tajo: how update (from S3) is triggered?

I set up a Tajo table from a S3 path and my understanding is that any changes in the S3 will be automatically applied to the Tajo table. Does Tajo poll status of the S3 object and see whether there is any change? How does that work in more detail? Thanks in advance!
kee
1

votes
1

answer
467

views

Conflict of sent_http_* logging and proxy_hide_header in nginx

I am trying to log some variables set by application to nginx log. So certain HTTP response headers are set and then I log them successfully with $sent_http_* variables. But then I don't want the info to be exposed to outside world so I tried to remove them by adding 'proxy_hide_header'. What happe...
kee
1

votes
1

answer
82

views

JSON uploading to Tajo

I have a flat JSON file which I am hoping to upload to Apache Tajo. I couldn't find JSON support in Tajo documentation. It is a flat JSON so I can just transform it as CSV but I am just wondering.
kee
1

votes
1

answer
88

views

Any Apache Tajo based SaaS to try?

Is there any Apache Tajo based service preferrably running out of AWS? I can easily set up a single node cluster for testing. But with multi-node cluster I can run more realistic testing and having some kind of Tajo SaaS would be so helpful.
kee
1

votes
1

answer
178

views

How to write a UDF in Tajo [closed]

I am wondering if I can write a UDF in Tajo especially in Python. My use case is for ETL where I want to group log records by some ID (browser ID), then sort the records in the same group by timestamp and then finally use my UDF to go over the sorted records in each group.
kee
1

votes
1

answer
188

views

How to set up Hive metastore off Redshift

I couldn't find a way to set up a metastore off Redshift for Hive. I am wondering if there is anyone who has tried this. Also since Redshift supports PostgreSQL, maybe it is possible. Please share if you have any experience. I am new to Hive and am using CDH5.4.
kee
1

votes
1

answer
2.9k

views

YARN error: TaskAttempt killed because it ran on unusable node … Container released on a *lost* node

I am using CDH 5.4 with Pig 0.12. I am getting a lot of this error from all nodes: TaskAttempt killed because it ran on unusable nodename:portnumber Container released on a *lost* node What does this mean? In particular what does 'lost' mean here? It doesn't look like the node is really lost in the...
kee
1

votes
3

answer
63

views

Swift 2: Why is it necessary to apply ‘as!’ after getting new object by ‘copy()’?

I’ve created new SKEmitterNode object using copy() method. After that i’ve tried to write emitter.position but Xcode said «Ambiguous reference to member ‘position’». But, when i use type conversion «as! SKEmitterNode» after the «copy()», everything is ok. Can you explain me, please, wh...
Kee Reel
1

votes
2

answer
72

views

How can I catch moment SKScene became paused or audioEngine became stopped?

How can I catch moment SKScene became paused or audioEngine became stopped? I have two SKScenes: GameScene and EndScene, and I play sounds during the game using audioEngine property of GameScene (this property contains AVAudioEngine object). When the game is over, the scene changes from GameScene to...
Kee Reel
1

votes
2

answer
1.9k

views

How to draw multi-lines from multiple queries in Kibana

I am new to Kibana and need some help. I can draw this line chart for a single query (java): Now I would like to another line for another query (for example python) in the same chart. I am not so sure how to do that. Also 'Markdown widget' is the way to add a legend? Any help would be appreciated.
kee
1

votes
3

answer
69

views

How to get key by counting values in dict?

I want to find the company that has made the most number of the worst cars ever made. output looks like this: Worst manufacturer: Ford Worst manufacturer: Ford Worst manufacturer: Triumph I tried this way: def print_worst_manufacturer(car_dict): for k,v in car_dict.items(): if len(v) > 2: print('Wor...
Kee
1

votes
1

answer
210

views

How to search Word2Vec or GloVe Embedding to find words by semantic relationship

Common examples of showing Word Embedding's strength is to show semantic relationship between some words such king:queen = male:female. How can this type of relationship be discovered? Is that through some kind of visualization based on geometric clustering? Any pointer will be appreciated.
kee

View additional