hlagvankar

6

votes
5

answer
474

views

Hive update with subquery

I'm trying to update a Hive table from subquery and I know hive doesn't support such updates. Is there any work-around for this? My update looks like this UPDATE tmp_aka SET guid = (SELECT mguid FROM tmp_maxs WHERE tmp_maxs.guid = tmp_aka.guid);
hlagvankar
1

votes
1

answer
63

views

Scala PLAY same routes

I have same routes in routes file but their action is different as shown GET /counts controllers.Application.getAllCountsByFeature(features) GET /counts controllers.Application.getAllCounts() I'm calling both routes as h...
hlagvankar
1

votes
3

answer
6.5k

views

Spark - How to update value using data frame Scala

I have two files with following structure File 1 gnk_id, matchId, timestamp File 2 gnk_matchid, matchid I want to update value of gnk_id in file 1 with value of matchid in file 2 if file1.gnk_id = file2.gnk_machid. For this I created two data frame in Spark. I was wondering whether we can update val...
hlagvankar
2

votes
1

answer
107

views

Snowplow spark - getting error at runtime

I'm parsing snowplow's events using Spark as per their guide at https://github.com/snowplow/snowplow-scala-analytics-sdk. My code looks like import com.snowplowanalytics.snowplow.analytics.scalasdk.json.EventTransformer import org.apache.spark.{ SparkConf, SparkContext } import org.apache.spark.Spar...
hlagvankar
2

votes
0

answer
211

views

s3distcp - takes long time to copy large number of small files from one bucket to another

I need to copy large number of small files from one S3 bucket to another. I'm using S3-Dist-Cp command provided by AWS. s3-dist-cp --src=s3://some-bucket/ --dest=s3://another-bucket/ --groupBy= --targetSize= --deleteOnSuccess Now, the problem with this command is that it takes forever to copy all sm...
hlagvankar
2

votes
1

answer
197

views

Inject database dependency scala Object

Is there any way to inject play database dependency in Scala object? I know we can do it for class like class MyClass @Inject() (db: Database) = { } but I want to inject dependencies without actually using Play plugin. My build.sbt looks like this scalaVersion := '2.11.8' lazy val sparkVersion = '2...
hlagvankar