Questions tagged [csv]

21619 questions
1

votes
3

answer
5.7k

Views

How to Open csv file with pandas data frame

There is a CSV format file with three column dataframe. The third column has long text. This error message occurred, when i tried to open the file using pandas.read_csv message : UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 0: invalid start byte. But there is no problem openi...
Antenna_
1

votes
1

answer
760

Views

AWS Sagemaker unable to parse csv

I'm trying to run a training job on AWS Sagemaker, but it keeps failing giving the following error: ClientError: Unable to parse csv: rows 1-5000, file /opt/ml/input/data/train/KMeans_data.csv I've selected 'text/csv' as the content type and my CSV file contains 5 columns with numerical content and...
swap709
1

votes
1

answer
392

Views

PowerShell Add-Content with multiple variables to columns

Using Add-Content to input multiple variables to a csv file will write each variable to the csv in a new row, such as the following: Code $name, $dateTime, $task | Add-Content $csv -Force Results $name $dateTime $task I would like to loop the above code with new variable values each time but would...
Garrett
1

votes
1

answer
982

Views

Export dataframe as csv file from google colab to google drive

i want to upload a dataframe as csv from colab to google drive.i tried alot but no luck.i can uploaded simple text file but but csv i tried following code. import pandas as pd df=pd.DataFrame({1:[1,2,3]}) df.to_csv('abc',sep='\t') auth.authenticate_user() gauth = GoogleAuth() gauth.credentials = Goo...
Athar Noraiz
1

votes
1

answer
21

Views

How to access nested JSON object in Python DataFrame

I have a JSON response that I am converting to a DataFrame in Python. JSON response: [ { 'id': 123456, 'first_name': 'John', 'last_name': 'Doe', 'fields': [ { 'title': 'ABC', 'value': '123' }, { 'title': 'DEF', 'value': '456' } ] } ] When I parse this JSON to a DataFrame, the columns appear as id, f...
Matt
0

votes
1

answer
18

Views

How do I allow pandas to process quote characters without recognizing them as EOF?

I'm trying to process a CSV file with pandas. One of my fields is book titles. Some of them have a comma in the title. I need to escape the comma with quotes in order for the INSERT statement to execute correctly in postgresql. However, when parsing the file, pandas sees '' in the last line as EOF....
trimonkeytri
1

votes
2

answer
41

Views

Pandas: split cell and append data to another cell

ok i am trying to split a cell in a csv and take the second half of the information and append it to another cells data. i can split the cell but then the data i want just vanishes. this code works to split the cells import pandas as pd data = pd.read_csv('/root/v-unit/calendar_data.csv') data['St...
stcoyle
1

votes
1

answer
34

Views

Exlude a row on Import-Csv

I have a question concerning Import-Csv in PowerShell. My script reads all *.csv files in my directory and writes it to my MS SQL database. So far so good, but I have a problem in these csv files. They all look the same like this: Header1;Header2;Header3 -----;-----;----- Data1;Data2;Data3 Data1;Da...
Patrick Sacher
1

votes
2

answer
63

Views

Converting a CSV file into appropriate XML format

I'm trying to get some tab-delimited CSV files into an XML via PowerShell, and I'm struggling to make sense of it all. I'm 100% sure it's something really simple I'm missing, so help would be massively appreciated. I'm very close but just can't get over the final hurdle. I essentially want this scri...
Honkytonk
1

votes
2

answer
61

Views

extract values of column according to value in another columns in csv file

Here is my file: id;verbatim;score 0;1; je suis beau;1 1;2; je suis laid;0 2;3;je suis merveilleux;1 3;4;je suis repugne;0 I would like to extract all sentences in column 'verbatim' which have a score of 1 and all which have a score of 0 so that I have two separated files: print(verbatim, score =...
kely789456123
1

votes
3

answer
138

Views

How can I pass a list comprehension results into a csv using python

I have two lists with variable lengths list1 = ['x1','x2','x3','x4','x5'] list2 = ['x5','x4'] I try the following to find the missing elements *[item for item in list1 if item not in list2], sep='\n' but if I do item = *[item for item in skuslist if item not in retaillist], sep='\n' csvwriter.writer...
Evridiki
1

votes
2

answer
29

Views

Saving each dataframe from a list to separate csv files

Im trying to save each element from a list to each separate csv files. each element is a dataframe. I used the following codes, however, the problem is that the files that it saves are only from the first or last element of the list from the two following codes respectively. e.g. the output files ar...
WDS
1

votes
1

answer
19

Views

Delete row: not matching with other columns

I have a dataframe looks like this: Type Brand A Brand B Type A ID A ID AA Type B ID A ID AA Type A ID B ID BB Type B ID B ID BB Type B ID C ID CC Type A ID C ID DD Type B ID D ID EE Type A ID D ID FF I want drop or eliminate rows data two brand column and ype co...
Arief
1

votes
2

answer
52

Views

Get record from csv by column name

I have a csv table (my csv file http://jpst.it/1GCd9) | id | email | name | google.com | yahoo.com | | 1 | [email protected] | jack | | + | | 2 | [email protected] | jack | + | | | 3 | [email protected] | jack | | | Is it possible to get emails that have '+' by column name? For example I wan...
user3514052
0

votes
0

answer
2

Views

How to plot data from a csv file with 7 columns and 20001 rows?

I have instruments that sample data a .00025 (quarter mil) rate. I typically record 10-20 seconds of data which gives me 40K-80K lines and 7 columns. I need to plot some of that data in wave form to illustrate displacement amongst other things. Need direction new to this MATLAB/Octave world. Examp...
Alexander Rivas
0

votes
1

answer
16

Views

Parsing csv-files, Store/write row to new file between to rows from old file

I'm trying to parse a CSV-file and store/write rows into a new CSV file between two specific rows from the old CSV file. Is it possible to store the interval rows between the two specific rows? For example: old.csv contains 1000 rows. the starting row is on the 63 rows in the old.csv file and the en...
snaz
0

votes
0

answer
5

Views

How to have a unique random number generated in a csv file every time without manual intervention?

I need to pass 10 digit alpha numeric random value (e.g. AUT******* where * denotes any random number) in a csv file. Contents of csv file: Code,Account1,Account2,Account3 AUT*******,ABC,DEF,GHI This csv file will be uploaded in to a web application every time to perform some operation. I need this...
Siva
0

votes
0

answer
14

Views

ADDING keys and value from a dictionnary to a csv file and filtered values of a dictionnary in python

I would like to add key and values of my dictionary : d_tag to my file csv starting from the third column , I tried a lot of solution but it doesnt work I have the Following error : Traceback (most recent call last): File 'CSV_dico.py', line 62, in writer.writerow(k) File 'C:\Users\Emmanuelle\A...
kely789456123
1

votes
1

answer
8

Views

fast-csv transform appears to be stripping columns

I am working on a project that at its base level parses a csv file and transforms it so that only certain columns are mapped to the new file using only certain columns from the parsed file. Including input .csv for reference. Co Code,Batch ID,File #,Reg Hours,O/T Hours,Reg Earnings,O/T Earnings,Hou...
mattymil
1

votes
2

answer
2.9k

Views

List filenames and folder paths on csv using batch file

I have a batch file that lists the full file directories (including filename at the end) onto a csv file. I need it to produce this, but also just the filename in a separate column. I would like it so that the format is Filename in first column (including extension) and full file directory in second...
user3361017
0

votes
1

answer
24

Views

Python CSV reader returns fewer rows than in file

I am struggling with the csv module. I have a sample CSV file that has 5000 lines (each line contains 7 values 0 or 1) with headers. I want to iterate through file in read mode and append file in write mode with new column values (prediction), but iteration stop after 478th row (like in sample code)...
1

votes
1

answer
917

Views

Backslash (\") issue with fgetcsv() while importing a csv file in php

Backslash (\') problem encounter while doing this example of csv import. here is my 2 input lines from csv 'Travel_and_Work_en.pdf',840722,3/22/2007 10:57 AM,10/10/2007 12:00 AM,10/10/2007 9:00 AM,'.pdf','e:\' '._.Trashes',4096,10/12/2010 4:55 PM,12/8/2014 12:00 AM,11/13/2014 3:00 AM,'.Trashes','e:\...
Sobin Augustine
0

votes
1

answer
23

Views

How to read rows from last to first?

I want read rows from last to first. But I can't do this. I want read in reverse order. stary_patient_2 ='Xs' nowy_patient_2 = 'VB' import csv with open(DATA_DIR+'/stage_2_train_labels_right.csv', 'r') as inp, open(DATA_DIR+'/stage_2_train_labels_right_poprawne.csv', 'w') as out: reader = csv.DictRe...
AlinaChuj
1

votes
2

answer
38

Views

How delete rows where `row[patient_id]` is duplicated and `row[Target]==0`?

I want delete all rows from my .csv file where: 1) row[patient_id] are duplicated 2) row[Target] == 0 Name of cols: patientId x y w h Target patientId x y w h Target '2341-vcb1' '10' '20' '30' '40' '1' '2341-vcb1' '70' '20' '70' '40'...
AlinaChuj
1

votes
3

answer
15.9k

Views

How to change file extensions in VBA

I feel like this must be simple, but I can't find the answer. I'm saving a bunch of csv files using vba and would like to change all the file extensions from .csv to .txt to import into another program (Revit) which only recognizes the .txt extension. Is this possible? Here is the command I'm usi...
user1707675
0

votes
1

answer
16

Views

No Output fast-csv writeToPath

I am writing a script which at its core parses a .csv file for certain columns storing them in an array and then writes the contents to another .csv file. I am able to parse the file using fast-csv and have confirmed in the terminal that my array is in the correct format. However, when I attempt to...
mattymil
1

votes
2

answer
5.6k

Views

How to map csv file to pojo class in java

I am using java maven plugin.I want to fetch employee.csv file records in pojo class. this pojo class I am generating from employee.csv header and all fields of pojo class are String type.now I want to map employee.csv to generated pojo class.my requirement is I dont want to specify column names man...
Chaitanya Ghumare
1

votes
1

answer
1.4k

Views

Gatling Feeder Issue : No attribute name 'CSVFieldName' is defined issue

I am newbie for Gatling and trying to read some fields from CSV and use them in my gatling scenario but facing No attribute name 'CSVFieldName' is defined issue ; some details: Gatling Version : bundle-2.2.3 CSV Name : memId.csv CSV contents : memid CKABC123 Scala File contents : //Class Decl...
Ritesh Sharma
1

votes
2

answer
1.1k

Views

How to read my csv file in Fortran?

How to read following data in Fortran. I'm not able to read data in this file. I'm getting following error severe (64): input conversion error, unit 18 Image PC Routine Line Source a.out 00000000004734AA Unknown U...
Ajay
0

votes
1

answer
20

Views

Parsing CSV file in vue/typescript

I am reading in a csv file and i am in the process of parsing through it but having some trouble. 1 - Before parsing the file i already have an array with strings of headers that i want to pull the data for from the csv file. 2- I want to parse the file so i can also display the data into a table...
bluePearl
0

votes
0

answer
8

Views

Issues breaking up huge CSV file using PowerShell

I am trying to break up a large CSV file into smaller ones. Original file: 5,055,000 lines of excel data. Desired file: 500,000 lines of data per file. However, there are 2 recurring errors that I absolutely cannot solve, no matter what. What I am trying to do, is open a CSV file, import up to 500,...
Electromagnetics
1

votes
2

answer
57

Views

Regular expressions python data extraction

How to use python regular expression to extract data from the below two string TASK000123-Tomcat server hosted on tbu.test1 is down-P1 --In Progress TASK000123-Tomcat server hosted on tbu.test1 is down-P1 --Completed I need the following csv file from this: Format: TaskID,Priority,Status TASK000123,...
Abeer Sheikh
1

votes
2

answer
76

Views

ArangoDB: How to export collection to CSV?

I have noticed there is a feature in web interface of ArangoDB which allows users to Download or Upload data as JSON file. However, I find nothing similar for CSV exporting. How can an existing Arango DB collection be exported to a .csv file?
BugBuddy
1

votes
5

answer
95

Views

Quickest way to organize categorized data in a text file and convert to CSV

I have a text file with hundreds of rows. Data fields and values separated by a colon and one empty line separating each data set. It looks something like this... icon:rain temperatureHigh:55.37 temperatureLow:42.55 humidity:0.97 windSpeed:6.7 precipType:rain precipProbability:0.97 icon:partly-cloud...
Johnny Carino
1

votes
2

answer
32

Views

How to create a for loop to open, mutate and save .csv files using R?

I have a several .csv files that have to be reformatted and saved again using an R script. The function that is needed to do the changes and the reformating of the files, is already established and works perfectly fine. But as there are always lots of documents to change, I would like to have a for...
Hotshot
1

votes
3

answer
162

Views

Get a massive csv file from GCS to BQ

I have a very large CSV file (let's say 1TB) that I need to get from GCS onto BQ. While BQ does have a CSV-loader, the CSV files that I have are pretty non-standard and don't end up loading properly to BQ without formatting it. Normally I would download the csv file onto a server to 'process it' and...
David542
1

votes
2

answer
43

Views

csv is writing all rows in one column python [duplicate]

This question already has an answer here: Writing Python lists to columns in csv 6 answers i am writing csv of my xml file which contain Date , ID , title , Abstract when i am running my code it is writing all rows in one column as list it not writing column separately it is writing like - ['123,...
Shayez Karim
1

votes
2

answer
32

Views

selecting by label and writing to csv

I am new to pandas so hoping this is an easy fix. I am trying to create a csv file that only contains the columns called out in df.loc[]. My csv file is currently showing all available rows/columns so the selection is not working. import requests, json import pandas as pd # url and date set earlier...
Bixby
1

votes
2

answer
33

Views

Not able to use for after using csv reader

I am given the task to write a script to check MX records of the given data in the CSV file. I have started by trying checking it using regex and before that I trying to read the CSV file. I would also like to log the progress so I am printing the row number it is on, but whenever I use the cvs_read...
Bhanu

View additional questions