Questions tagged [google-cloud-datastore]

0

votes
4

answer
428

Views

Explain the functionality of JSON

I think it is better to understand why I have so many problems with JSON that I explain you what my goal is: I work with Googles App Engine. There I want to store data. The data looks like user - username question - question date1 - date1 date2 - date2 An Android App have the 'simple' function to: S...
StefMa
1

votes
1

answer
705

Views

How do I find a max value in datastore?

I am using go and datastore and I want to have a query where I get the max value from the datastore. I can't find anything about max values in the docs. How can I accomplish that? I am using Google App Engine https://cloud.google.com/appengine/docs/go/datastore/
theva
1

votes
2

answer
711

Views

Storing a string from a textbox to the Google App Engine Datastore

Is it possible to create a textbox in HTML, type a string into it, click a 'save' button, store that information onto a GAE datastore model and have the text stay displayed in the textbox and saved in the datastore? The HTML is on a separate file, just rendered through my main.py file using class M...
tomzaragoza
1

votes
0

answer
217

Views

GO Datastore connection reuse

I am new to GO and i need to reuse the datastore client in the entire application instead of creating it again and again for each API call. I have created a db package and in it Init i am storing client in a global db variable to reuse it via a GetDB() function. But i am getting the below error Per...
Vishal Bisht
1

votes
1

answer
20

Views

Is there any limit in list when applying entity filter with “in” operator

I need to apply 'in' filter. I want to ask is there any limit how much element need to insert in list. For example List list; // There is at least 1500 items in this list ofy().load().type(Entity.class).filter('property in', list); Can I apply as much element as I want or is there any limit or...
Arfan Haider
1

votes
1

answer
50

Views

Kindless ancestor query for different entity kinds in golang

According to the documentation, it should be possible to retrieve an ancestor and all of its descendants, regardless of their kind. In my implementation, I have a different kind of ancestor and descendant. The following codes however always returns the error 'invalid entity type': q := datastore.New...
fsapei
1

votes
0

answer
95

Views

Is the schema needed to deserialize protobuf bytestring?

I'm trying to deserialize some Protobuf bytestrings, in Python, outside of GAE(AppEngine) environment. I'm wondering if I need the message schema for this use case. Approaches I've tried so far includes google.cloud.proto.datastore.v1.entity_pb2.Entity.ParseFromString(): >>> pb = ... ## fetched a...
xiaolong
1

votes
2

answer
86

Views

Google Datastore Transaction API vs. Datastore API when in a transaction

I have been battling Google Cloud Datastore transactions for two days. I am unsure whether or not I should use datastore.save or transaction.save when within a Google Datastore transaction. Common code: const datastore = new Datastore(); const transaction = datastore.transaction(); Scenario 1: tran...
jcgh582
1

votes
1

answer
208

Views

Memory limit exceeded error when reading a large file with node js and readline

I am getting a memory error when using readline. The section of code is shown below: var lineReader = require('readline').createInterface({ input: require('fs').createReadStream('/tmp/temp.ttl') }); let entity; let tripleKey; let triple; console.log('file ready for processing'); lineReader.on('line...
user2302244
1

votes
0

answer
34

Views

How to generate User specific feeds?

I have 2 tables - Pages and Feeds. Each feed belong to a specific page. The user follows the interested page. My requirement is to provide feeds for the user from the pages he has followed in a random way. That is, if page1 has 5 feeds & page2 has 5 feeds, I would like display these 10 feeds in a...
learner01
1

votes
1

answer
80

Views

Google App Engine Datastore - query with StructuredProperty in projection and filter

I have a couple ndb models looks like these: class Product(ndb.Model): manufacturer = ndb.StringProperty() category = ndb.StringProperty() price = ndb.FloatProperty() class Customer(ndb.Model): customerId = ndb.StringProperty() name = ndb.StringProperty() products = ndb.StructuredProperty(Produc...
James
1

votes
1

answer
290

Views

Return entity key from datastore

I would like to know if there's a way to return a key from entity specific using nodejs and datastore. Example: I have an entity that has name and age parameters. I would like to retrieve the key by name. How could I do this?
Fernando Antunes
1

votes
0

answer
352

Views

cannot find module 'google-module'

I'm excited to get started with the Particle Electron logging GPS coordinates and storing them up in the Google Cloud Platform. I have all the integrations connected using this tutorial, https://docs.particle.io/tutorials/integrations/google-cloud-platform/ l and am now getting this error trying t...
1

votes
1

answer
32

Views

Does Objectify have an equivalent of Datastore.add() to insert non-existing entities only?

I would like to insert multiple entities (under the same entity group) into Datastore as a batch and only have the missing ones inserted and the rest unmodified. Datastore.add(Entities...) seems to support it as explained in this client issue and in the docs. I dont see an alternative on ofy() as t...
noobNeverything
1

votes
1

answer
34

Views

Google NDB Model syntax

I have an example line of code for google ndb model (datastore) V=model.Video.query(model.Video.key==key).get() Assuming that the 'key' has a value, is the instantiation lines below feasible/valid or syntactically correct ? + (Video)key.get() + V = model.Video(key) Not sure if they would work. Thank...
Renzo Rique
1

votes
0

answer
95

Views

GAE with datanucleus and JDO - fails to test locally - Getting javax.jdo.JDOUserException: Persistent class has no table in the database

I am refactoring a GAE application which I developed few years ago (and works fine) to use Maven and Java 8. The application uses JDO & datanucleus. The app is being built with no errors and I can browse to the home page, but I encounter the following error when trying to access the database (for...
1

votes
0

answer
219

Views

Python: google-cloud-datastore in AWS Lambda

Because Google Functions does not support Python, I was hoping to use google-cloud-datastore in AWS Lambda but hit the same error as AWS Lambda to Firestore error: cannot import name 'cygrpc' google-cloud-storage works just fine in Lambda so core packaging is not the issue but could pip'ing datasto...
1

votes
0

answer
145

Views

Read flattened entity from cloud datastore in golang

func (db *dataStore) AddAcceptance(ctx context.Context, req *acceptance.PolicyAcceptance) (uint64, error) { accpKey := datastore.IncompleteKey('Acceptance', nil) key, err := db.Put(context.Background(), accpKey, req); if err != nil { log.Fatalf('Failed to save Acceptance: %v', err) } accpKey = key...
ayush gupta
1

votes
0

answer
152

Views

no matching index found. recommended index is

I am really confused by how to solve this issue. I tried all the answers I could find from googling this issue. I got this error, as many other people got. no matching index found. recommended index is:\n- kind: Task\n ancestor: yes\n properties:\n - name: appName\n - name: RequestTime\n dir...
Chris L
1

votes
4

answer
1.4k

Views

How to make http call on DialogFlow v2 using Javascript ajax call

I found this example on the official site of DialogFlow using Node.js and it is working fine, but I dont know how do I integrate this into my web application. Is it possible that I can integrate this into my other javascript jquery code? and here I need to run node index.js but do I still need to do...
Mizlul
1

votes
1

answer
79

Views

Google Datastore Order with ancestors

Language: Python. I'm using datastore python lib, everything works fine. When using datastore query to sort the query result, I can add query.order = ['name']; to sort the result. But when query a table with ancestors, like: ancestor = client.key('user', name, namespace = NAMESPACE) query = client.q...
Dilemma
1

votes
1

answer
76

Views

How to convert Datastore Key to URL-safe format using PHP client library?

I'm trying to obtain a Google Cloud Datastore URL-safe key using the PHP Google Cloud Client Library. The java client library provides the functions toUrlSafe and fromUrlSafe to serialize a Key object from/to an URL-safe string. What are the php equivalents?
m7913d
1

votes
2

answer
93

Views

How to periodically update a moderate amount of data (~2.5m entries) in Google Datastore?

I'm trying to do the following periodically (lets say once a week): download a couple of public datasets merge them together, resulting in a dictionary (I'm using Python) of ~2.5m entries upload/synchronize the result to Cloud Datastore so that I have it as 'reference data' for other things running...
Grey Panther
1

votes
2

answer
37

Views

Trouble retreiving an entitiy from Google Cloud Datastore by Key in Google Cloud Functions

I have an entity set up with a name identifier: Namespace: [default] Kind: Cocktail Name: martini I have a Google Cloud Function: // Imports the Google Cloud client library const Datastore = require('@google-cloud/datastore'); const projectId = 'my-project-name'; const datastore = Datastore({ projec...
Jeremy
1

votes
2

answer
54

Views

Optimize Cloud Datastore NDB query to reduce read operation counts

After running my GAE app for 1 week, I got 30 millions count and I have to pay 23.31 usd :( Cloud Datastore Read Ops Japan 30,245,982.00 Count $23.31 Please help me to optimize queries. In my app, I have a lot of queries, mainly to check if entity does not exist, then do something tweet_db_qry = mo...
John
1

votes
0

answer
192

Views

How to get entity id from entity data with Google Cloud Datastore?

Here is the output I am getting from Google Cloud Datastore query in NodeJS: { photo: null, login: '[email protected]', [Symbol(KEY)]: Key { namespace: undefined, id: '5637017136267264', kind: 'Users', path: [Getter] } } I want to get [Symbol(KEY)].Key.id value in NodeJS. I am using gstore-node Plea...
Piyush Bansal
1

votes
1

answer
50

Views

Querying datastore with NDB

I'm trying to use Python and NDB to access the datastore, which contains one entity: I've defined by NDB model with the following code: class Test(ndb.Model): name = ndb.StringProperty() val = ndb.IntegerProperty() Then I run a query for the entity: query = Test.get_by_id('Testing') This returns a N...
MatthewScarpino
1

votes
0

answer
417

Views

cloud functions python to access Datastore

I am looking for a tutorial or document on how to access datastore using cloud functions (python). However, it seems there is only tutorial for nodejs. https://github.com/GoogleCloudPlatform/nodejs-docs-samples/tree/master/functions/datastore Can anybody help me out? Thanks
John
1

votes
1

answer
48

Views

How to fetch the indexes/Columns data from Google Cloud Platform using GQL(Google Query Language)

Kind name: invoice_header Name/ID :id=4829628648652800 amount :2900 booking_ids :1,2 created_at :2018-09-04 10:20:30 discount_amount :23 due_amount :9999 indicator...
Narasimharao Swayampu
1

votes
0

answer
70

Views

How to query multiple namespaces using beam ReadFromDatastore

My pipeline produces a Pcollection with unique_namespaces that I want to pass as a query parameter to ReadFromDatastore and then Flatten all the reads. It looks like it is not possible to pass namespaces iterative list as a side input coll = (p | 'Reading from Datastore' >> ReadFromDatastore(projec...
Andrey
1

votes
1

answer
73

Views

How can I get total records count from Google Datastore using Gstore-Node?

I am using gstore-node package for querying data from Google Datastore. I did not find anywhere how to get the total record count of the query using gstore-node. Please help me out if anyone knows. Thanks in advance.
Piyush Bansal
1

votes
1

answer
91

Views

Datastore client query.fetch() throwing up a error

i am getting this exception when trying to create a new entry into Users tables. Currently Users table is empty.Basically i am querying the datastore kind if the record exist.For some reason it is throwing exception at the below code line result = list(query.fetch()) Complete Code class User(object)...
user7987134
1

votes
1

answer
40

Views

Database dump and restore from AWS postgres-9.6.6 database to Google Cloud SQL (also postgres-9.6)

I've dumped my database as described in Exporting data from an externally-managed database server: pg_dump -U [USERNAME] --format=plain --no-owner \ --no-acl [DATABASE_NAME] \ | sed -E 's/(DROP|CREATE|COMMENT ON) EXTENSION/-- \1 EXTENSION/g' > [SQL_FILE].sql The database I'm dumping from is running...
martins
1

votes
1

answer
211

Views

403 Missing or insufficient permissions

I am running datastore emulator. When I run 'dev_appserver.py app.yaml' command, I get this error 403 Missing or insufficient permissions. This is the warning I get. I know there is a problem with authentication. I have gone through this. But couldn't end up finding a solution. Some details: I am us...
Salman Haseeb Sheikh
1

votes
0

answer
26

Views

CLoud Datastore: Transactions implementation details and monitoring

I've recently started to use transactions. However I'm missing some pieces of documentation. Up till now I had a regular save and get functions which were calling the datastore. I was measuring these ones for latency. When working with transactions when do I actually hit the network in the followi...
Dan Dinu
1

votes
0

answer
31

Views

Bulk get using objectify?

I am using the Objectify Loader ids method to batch load the ids from the google datastore. But from the Google Cloud Trace logs, I can see that this requests are batched in ids of size 10. Is there any way to increase this size? However, if I change the consistency level of objectify to Eventual an...
Kishu Agarwal
1

votes
0

answer
31

Views

Why is there a 25 entity group limit for transactions in Cloud Datastore?

I understand why there is a write throughput limit of 1 second per entity group (strong consistency requirements for replication, etc). I don't understand why you are limited to 25 different entity groups in a transaction. Presumably this is for some other consistency/performance reason but I am hav...
Andrew
1

votes
0

answer
27

Views

Google Datastore client query similar to SQL union or In-clause

I'm exploring ways (Since I'm new to google datastore) to achieve bulk request similar to what we do using SQL IN-Clause or Union. What I currently have (below) is very inefficient way of executing query per item in for loop. Any help is much appreciated. I need to do this in Flex-Environment, means...
RKuyya
1

votes
1

answer
50

Views

Django app works locally but deployment on GCP say 500 Server Error, “ImportError: Could not import settings”

On GCP dashboard I get this error but on my machine all works fine: ImportError: Could not import settings 'site.settings' (Is it on sys.path?): No module named settings at __init__ (/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/third_party/django-1.4/d...
user10603251
1

votes
1

answer
67

Views

GQL Query Limit

I have created a NDB Model class Account(ndb.Model): accId = ndb.StringProperty() firstName = ndb.StringProperty() lastName = ndb.StringProperty() So now I need to query on this model and get account Id's of all people having firstName as Mark. So I wrote the query... acc_obj = Account.gql('WHERE fi...
skaul05

View additional questions