Questions tagged [ml]

1

votes
0

answer
11

Views

How exactly tf.estimator.DNNClassifier calculate Output/Logits?

i am new in Machine Learning here, just try to experimenting with tensorflow to understand some fundamental concept. I want to ask about how exactly tensorflow calculate output/logits layer, especially with Pre-Made models like tf.estimator.DNNClassifier. When i try to see the logits value from tens...
1

votes
1

answer
2.2k

Views

Customize Distance Formular of K-means in Apache Spark Python

Now I'm using K-means for clustering and following this tutorial and API. But I want to use custom formula for calculate distances. So how can I pass custom distance functions in k-means with PySpark?
Phong Pham
1

votes
1

answer
558

Views

Spark (Java) transformSchema() in custom Transformer

I want to use my custom transformer along with StandardScaler: VectorizerTransformer vectorizerTransformer = new VectorizerTransformer(field.getName()); pipelineStages.add(vectorizerTransformer); StandardScaler scaler = new StandardScaler() .setInputCol(vectorizerTransformer.getOutputColumn()) .setO...
Igor Kustov merge me
1

votes
2

answer
324

Views

Error in Tensorflow Object Detection running on Cloud ML: no module object_detection.train

Not sure if this is a bug (file report), or I did something wrong. System information: Linux 17.04 TensorFlow version: 1.9.0 Python version: 2.7.13 Command I used: gcloud ml-engine jobs submit training object_detection_$(date +%Y%m%d_%H%M%S) \ --job-dir="gs://mybucket/train" \ --packages dist/obje...
1

votes
1

answer
39

Views

How to tune hyperparameters using custom model evaluation algorithm?

In my model evaluation algorithm I would like to get model predictions for validation data and apply an algorithm which models and imitates some real-world scenarios based on validation data and predictions. In my scenario evaluation algorithm depends not only on true target values (y_true) and pred...
roman
1

votes
2

answer
67

Views

Access from gcloud ml-engine jobs to Big Query

I have a python ML process which connects to BigQuery using a local json file which the env variable GOOGLE_APPLICATION_CREDENTIALS is pointing to (The file contains my keys supplied by google, see authentication getting-started ) When Running it locally its works great. Im now looking to deploy my...
1

votes
1

answer
28

Views

How to deploy templates to different schema databases?

I have staging and final databases, both currently sharing a schemas database. We're using TDE, where some templates apply to staging and some to final. Can ml-gradle deploy some templates to one schemas database and others to a second?
Dave Cassel
1

votes
1

answer
186

Views

Defining a normal form with Haskell

I'm trying to get an expression and turn it into a standard form. To better clarify my purpose, assume you define a general style for your expressions like this: ∑(a*b) // sum of products Now if you're given an input which is not in that format like: (a+b)*(c+d), you'll need to normalize it firs...
ye9ane
1

votes
1

answer
144

Views

(Beginner's) issue with redundant case statement in SML

I'm trying to write a function in SML to compute the partial sum of an alternating harmonic series, and for the life of me I can't figure out why the compiler says one of the cases is redundant. I haven't used case statements before(or local, for that matter), but the order of these cases seems rig...
xhevahir
1

votes
1

answer
3.2k

Views

ERROR: Inserting EQUALOP

Hi can someone help me figure why i get the ERROR: Inserting EQUALOP for the following code? fun generator inchan outchan n = if n>0 then (let fun loop () = val c = recv(outchan) val _ = send (inchan, c) in (loop ();(generator inchan outchan (n...
1

votes
1

answer
212

Views

how to typecast a value in ocaml

I am trying to typecast a value from one type another. type days = | Mon | Tue | Wed | Thu | Fri | Sat type ('a, 'b) holidays = H of 'a | W of 'b (* H: holiday, W: Working *) let my_list = [H Mon; H Tue; W Wed] From the above my_list, I would like to extract, "Mon" and not "H Mon". How can I go abou...
recursion1212
1

votes
1

answer
89

Views

Can you do typecasting in ML?

I haven't done any programming in ML, but I am studying functional programming. ML is really particular about types in general, but I wanted to know if you could do typecasting to get around its strict type limitations? If you can't, is it because ML wants to lessen the chances of incorrect types, o...
JustBlossom
-1

votes
1

answer
89

Views

Update Machine Learning models in Mllib dataframe based PySpark (2.2.0)

I have built a Machine Learning model based on clustering, & now just want to update it with new data periodically (on daily basis). I am using PySpark Mlib, & not able to find any method in Spark for this need. Note, required method 'partial_fit' available in scikit-learn, but not in Spark. I am no...
bioinformatician
0

votes
0

answer
8

Views

Keras Load_module : Unknown Layer:Name

The following is my model that I am using when training model = Sequential() model.add(Conv2D(64, kernel_size=3, use_bias=False, input_shape=(size, size, 1))) model.add(BatchNormalization()) model.add(Activation("relu")) model.add(Conv2D(32, kernel_size=3, use_bias=False)) model.add(BatchNormalizat...
BearsBeetBattlestar
1

votes
0

answer
10

Views

UWP: The type or namespace name 'TensorFloat' could not be found

I am trying to run a UWP app I made (which worked) on a different computer, and I am now getting the error: The type or namespace name 'TensorFloat' could not be found (are you missing a using directive or an assembly reference?) I have VS 2017, my Windows 10 version is 1809, build 17763.194, and I...
Eutherpy
0

votes
0

answer
2

Views

ML.NET - Multiclass Classificaion score values

I currently have a project to take large bits of text and classify them as types. This is similar to the sentiment sample provided by microsoft except its multiclass instead of binary. I have the code working just fine and will likely become stronger as we add data to it. However, i have hit a snag...
Brendt McKnight
2

votes
1

answer
66

Views

Getting worse results on gcloud vs local training

I'm trying to understand why my local results are better than the gcloud results. In local, I run a job like this: gcloud ml-engine local train --module-name trainer.task --package-path trainer -- --vocabulary-file trainer/data/vocab.txt --class-files $CLASS_FILES --job-dir trainer/lr0001 --num-epoc...
djma
2

votes
1

answer
40

Views

int -> ('a -> 'a)

I have a question that I came across studying SML. It requires the return type to be int -> (’a -> ’a). This is what I have done: - fn x:int => fn y => y; val it = fn : int -> 'a -> 'a How can I make the return type to be int -> ('a -> 'a) where 'a -> 'a is in brackets? Any help would be appreci...
Naeem Khan
2

votes
2

answer
97

Views

StandardML 97 definition

Actual Standard ML specification is The Definition of Standard ML (Revised) , which available on MITPress only in a print version. Is it availaible in electronic format (pdf, ps, i.e.)?
S.J.
9

votes
1

answer
494

Views

Spark ML - MulticlassClassificationEvaluator - can we get precision/recall by each class label?

I am doing a multiclass prediction with random forest in Spark ML. For this MulticlassClassificationEvaluator() in spark ML, is it possible to get precision/recall by each class labels? Currently, I am only seeing precision/recall combined for all class together.
Sam
3

votes
2

answer
1.3k

Views

Does Spark.ml LogisticRegression assumes numerical features only?

I was looking at the Spark 1.5 dataframe/row api and the implementation for the logistic regression. As I understand, the train method therein first converts the dataframe to RDD[LabeledPoint] as, override protected def train(dataset: DataFrame): LogisticRegressionModel = { // Extract columns from d...
Nikhil J Joshi
3

votes
3

answer
1.2k

Views

How to Split the Predicted Probabilities Produced by ML Pileline Logistic Regression

I'm trying to extract the predicted probability from the logistic model using ML pipeline and DataFrame API. The output of predicted probabilities is a column vector that stores the predicted probabilities for each class(0, 1) in as shown below. I wonder how I can extract only the probability for cl...
SH Y.
2

votes
1

answer
531

Views

RPAREN EQALOP and unbound variable and constructor errors in SML/ML

Said I have got 2 CNF logical phrases a,b and my distrib function should return the CNF form of a|b (a OR b). Replacing rules that I've got are: 1) Replace p|(q&r) by (p|q)&(p|r) 2) Replace (q&r)|p by (q|p)&(r|p) A prop defined this way: datatype prop = Atom of string | Not of prop | And of prop*p...
user550413
2

votes
1

answer
258

Views

Returning a recursive function in ML

How can I return a recursive function in ML? As far as I know recursive anonymous functions cannot be returned and only anonymous functions can be used as returning value(if the returning value is function...).
poli
1

votes
2

answer
4.2k

Views

Logistic regression with spark ml (data frames)

I wrote the following code for logistic regression, I want to use the pipeline API provided by spark.ml. However it gave me an error after I try to print coefficients and intercepts. Also I am having trouble computing the confusion matrix and other metrics like precision, recall. #Logistic Regress...
Shweta Kamble
1

votes
1

answer
123

Views

Type clash in Standard ML datatype with same constructor name

I need to describe in Standard-ML a language made of properties and values. My property system is made of properties which can have values, like for example: color: red | yellow | blue | transparent align: left | center | right bgcolor: red | yellow | blue | transparent I created this sml file which...
Andry
1

votes
3

answer
246

Views

ML list typing when converting char list to int list

I've been having an issue with converting a character list to an int list. My goal is to basically take a number such as 325 and have it return a list of [3,2,5]. What I've done so far is take the number and then convert it to a string, which then gets exploded to a char array. I then want to conver...
Matt Matero
1

votes
2

answer
115

Views

Why automatic currying in ML family languages

ML family languages have automatic currying - where a function call with two arguments is considered to apply them one at a time, f x y = (f x) y - whereas other language families don't. Is this just a historical accident, or is there a technical reason for this? Note that I am not asking whether au...
rwallace
16

votes
6

answer
637

Views

Can compilers deduce/prove mathematically?

I'm starting to learn functional programming language like Haskell, ML and most of the exercises will show off things like: foldr (+) 0 [ 1 ..10] which is equivalent to sum = 0 for( i in [1..10] ) sum += i So that leads me to think why can't compiler know that this is Arithmetic Progression and use...
Nyan
17

votes
3

answer
7.3k

Views

What is the difference between HashingTF and CountVectorizer in Spark?

Trying to do doc classification in Spark. I am not sure what the hashing does in HashingTF; does it sacrifice any accuracy? I doubt it, but I don't know. The spark doc says it uses the "hashing trick"... just another example of really bad/confusing naming used by engineers (I'm guilty as well). Coun...
Kai
17

votes
4

answer
6.9k

Views

Dropping a nested column from Spark DataFrame

I have a DataFrame with the schema root |-- label: string (nullable = true) |-- features: struct (nullable = true) | |-- feat1: string (nullable = true) | |-- feat2: string (nullable = true) | |-- feat3: string (nullable = true) While, I am able to filter the data frame using val data = ra...
Nikhil J Joshi
17

votes
2

answer
12.6k

Views

How to cross validate RandomForest model?

I want to evaluate a random forest being trained on some data. Is there any utility in Apache Spark to do the same or do I have to perform cross validation manually?
ashishsjsu
1

votes
2

answer
698

Views

SML fibonacci large numbers

I tried to write my own fib function that works for large numbers (over 50) and I had no luck. First I tried the obvious solution but that overflows way to quicly. my next solution was this $fun fib(a:int, b:int, index:int) = if(index = 1) then $ (a+b) $...
Katianie
17

votes
4

answer
6.5k

Views

Google Dataflow vs Apache Spark

I am surveying Google Dataflow and Apache Spark to decide which one is more suitable solution for our bigdata analysis business needs. I found there are Spark SQL and MLlib in the spark platform to do structured data query and machine learning. I wonder is there any corresponding solution in the Go...
Browny Lin
1

votes
2

answer
276

Views

Evaluator in OCaml/ML

type bool_exp = TT | FF | Var of string | And of bool_exp * bool_exp | Not of bool_exp ;; eval : bool_exp -> (string -> bool) -> bool I am trying to write an evaluator function called eval. I am very new to OCaml and not used to the syntax. Where can I start in writing this?
SpruceTips
2

votes
1

answer
294

Views

PySpark read in a textfile as Dense vectors

I'm working with PySpark and trying to load a file of the format below, where each line is a vector of counts [1394, 56692, 0, 10, 22] [0, 0, 0, 0, 0] [2235, 123, 678, 0, 999] And I'm trying to load this into Spark using sc.textFile(path/to/counts.txt). How can I convert each row of the file to be a...
Patrick Ruff
1

votes
1

answer
145

Views

Type clash when creating a list from a set abstype in SML

I feel like this should be easier than I'm making it, but I'm having some trouble with this function that should change a set to a list. fun explode(st(x, xs)) = let val listTemp = [] in if isEmpty(xs) then listTemp::x else explode(xs)::x end | explode(nilset) = []; I'm getting a type clash, saying...
Ross Verry
1

votes
4

answer
723

Views

how can i find partitions of given list? [closed]

how can i find all partitions of list of integers? mostly i need algorithm that uses recursion as i'm gonna implement it in SML. i need just algorithm, i will do coding part by myself. by mistake i wrote code for finding subsets and i don't have too much time left for this SML is kinda similar pasca...
unknown
1

votes
1

answer
1.2k

Views

How to get a string from TextIO in sml/ml?

I'm trying to read text from a file in SML. Eventually, I want a list of individual words; however, I'm struggling at how to convert between a TextIO.elem to a string. For example, if I write the following code it returns a TextIO.elem but I don't know how to convert it to a string so that I can c...
Nosrettap
1

votes
1

answer
2.2k

Views

SML Sum of List using option and Pattern matching

I'm creating sum of list and using option in it. When I pass an empty list, I should get NONE or else SOME value. I'm able to do that in the following way: fun sum_list xs = case xs of [] => NONE | x => let fun slist x = case x of [] => 0 | x::xs' => x + slist xs' in SOME (slist x) end But I want...
Sibi

View additional questions