Questions tagged [csv]

14133 questions
0

votes
0

answer
18

Views

I have a csv file of 11GB, which I'm not able to open in excel or in python using pandas library. Can anyone suggest some ways to split the csv file?

when used the pandas library, it showed an error: Initializing from file failed. Currently using windows platform
sb96
0

votes
0

answer
14

Views

How do I save a retrieve specific data from a csv file without headers in java?

I am writing an application which needs to load a large csv file that is pure data and doesn't contain any headers. I am using a fastCSV library to parse the file, however the data needs to be stored and specific fields need to be retrieved. Since the entire data is not necessary I am skipping ever...
Pavlacc
1

votes
2

answer
51

Views

PHP str_getcsv() does not parse CSV correctly if it contains Japanese character

I am trying to convert an excel file to an array using file() function. Some fields are containing Japanese character. For those field, I am not getting correct data. Here is my code line $data = array_map('str_getcsv', file($path));
Sojol Haque
-1

votes
1

answer
18

Views

How to change a .csv into lists and vice versa in python?

I need to split a .csv file: user1,pass1,email1 user2,pass2,email2 user3,pass3,email3 into 3 lists that can be called upon and appended too seperately. with open('userdata.csv', 'r') as f: userlist = [row[0] for row in csv.reader(f)] passlist = [row[1] for row in csv.reader(f)] emaillist = [row[2] f...
Hunter Guimont
1

votes
0

answer
20

Views

How do I export multiple header rows in Taleo Connect Client?

I am using Taleo Connect Client to export a list of candidates from Taleo to a CSV file. I want to format my CSV file to start with three header rows: A description, variable name, and data type. CandidateNumber,Country,StartDate CANDIDATE_NUM,COUNTRY_CODE,EFFECTIVE_START_DATE VARCHAR2,VARCHAR2,DATE...
Steven M. Vascellaro
1

votes
0

answer
70

Views

Loading csv with non-standard separator into BQ

Suppose I have the following data in a csv file: '"tom","jones","hello,\nMy name is tom"\x01\n"sarah","smith","hello"\x01\n' The line terminator is \x01\n. Would it be possible to load this directly into GCS (without first pre-formatting it)? My thinking process was to do: Load it into CSV using a n...
David L
0

votes
0

answer
4

Views

How to write a CSV file from a dictionary and populate the data into individual rows, rather than all into the same cell

I am trying to write data from a dictionary into a CSV file with each column corresponding to a key in the dictionary. My first column properly has the header for the key, but all of the data for each key ends up contained as a list within a single cell under the corresponding key. How do I get th...
Andrew
0

votes
1

answer
19

Views

How to send the value of variable from a child component to the parent component?

I need to load data from CSV file (see CSVDataTable.js provided below) and pass these data to the main component BatchFlights.js in the variable this.state.csvData. Then I use this.state.csvData in the API request in fetchData(). The component CSVDataTable.js loads data correctly. However, how can I...
ScalaBoy
0

votes
0

answer
4

Views

Reading Csv file written by Dataframewriter Pyspark

I was having dataframe which I wrote to a CSV by using below code: df.write.format("csv").save(base_path+"avg.csv") As i am running spark in client mode, above snippets created a folder name avg.csv and the folder contains some file with part-* .csv on my worker node or nested folder then file part-...
Ayush Mishra
-1

votes
1

answer
22

Views

Print data in a single csv file having different lengths using python

I have two files, one is a data frame while the other one is a list. I want to print both the files side by side so that a comparison can be done. The output should be like: Col1 Col2 Col3 A B C and so one. The first file consists of 14 columns while the second file which is a list c...
Dustrokes
0

votes
1

answer
22

Views

Python Pandas: Group files in a directory by similar filenames and concatenate dataframes in a specific order

My goal is to group .csv files in a directory by shared characteristics in the file name. My directory contains files with names: After_Source1_Receiver1.csv After_Source1_Receiver2.csv Before_Source1_Receiver1.csv Before_Source1_Receiver2.csv During1_Source1_Receiver1.csv During1_Source1_Receiver2....
Colton Medler
0

votes
0

answer
4

Views

Download csv from a password protected website (R)

If you go to the website https://www.myfxbook.com/members/iseasa_public1/rush/2531687 then click that dropdown box Export, then choose CSV, you will be taken to https://www.myfxbook.com/statements/2531687/statement.csv and the download (from the browser) will proceed automatically. The thing is, you...
Denny Seccon
0

votes
0

answer
15

Views

Unable to write the data into newline in csv file for dataframe

Hello guys I have a data which in datadrame i want to dump that data into csv file but i'm getting the all the data in one line i'm not getting how to make it happen please guys its a request don't make it duplicate please answer it where i'm doing wrong this is my data: Asset_Id Asset Family...
rahul singh
1

votes
1

answer
318

Views

reading a complicated CSV file with python

I didn't know how to title this question so if it should be renamed please inform me and I will. I'm reading a csv file that I saved off from a piece of equipment I used in taking some measurements. The data and various other key information is all being saved off. I've worked all day long on this a...
SCY
1

votes
1

answer
358

Views

Writing on the topmost row of a CSV file

I have this sample.csv file: a 1 apple b 2 banana c 3 cranberry d 4 durian e 5 eggplant And have the following code: samplefile = open(sample.csv,'rb') rows = samplefile .readlines() outputfile = open(output.csv,'wb') wr = csv.writer(outputfile) for row in rows: wr.writerow(row)...
python novice
1

votes
2

answer
2.9k

Views

Export Access Query WITHOUT Formatting

Relatively simple, but I can't seem to work it out. I want to export a query from access into a .csv (tab or comma delimited). When I do it manually through the wizard it works fine. But when I do it via vba, it comes complete with dash formatting that looks like the borders in the table! I tried t...
REdim.Learning
0

votes
2

answer
20

Views

Does anyone know how you can add a Title to a .csv file with Oracle?

I have some .csv files I'm outputting to email for reporting purposes. I'm happy with that. BUT, I want to make the reports a bit more readable by adding a "Title" This is just an example of the column structure currently... i'd like to add something before all this that has "Title" create or replac...
Dave
1

votes
1

answer
413

Views

Composing a line reader from a buffered stream using python io

I am using python boto to interact with s3. The files I have on s3 are CSV's where I'd like to read lines from s3 using a buffer to bound memory usage. I was wondering if anyone had any way of composing python's io classes to achieve this? The goal is to have some sort of abstraction that is able...
dm03514
0

votes
1

answer
25

Views

edit code to stop downloading on a certain month AND year

I am currently using the following code to download some data. The code works well except when I want to stop the downloading on a certain month of a year. E.g. If I want to stop downloading on month=02 and year=2019, it doesnt't work. I set max month to 02 and max year to 2019, however then it onl...
user8261831
0

votes
0

answer
9

Views

Saving streamed Data from Unity in a csv File

for a schoolproject I have to record the coordinate-data (vector of x,y,z) of an eye-tracking System and save it, for later evaluation. The whole eye-tracking System is integrated inside a Head-mounted-Display and the software runs over Unity. After some research I figured out that saving the Data...
Afrobeta
1

votes
2

answer
2.1k

Views

Reading a part of csv file

I have a really large csv file about 10GB. When ever I try to read in into iPython notebook using data = pd.read_csv("data.csv") my laptop gets stuck. Is it possible to just read like 10,000 rows or 500 MB of a csv file.
John Constantine
-1

votes
1

answer
28

Views

How do I check to see if any of the words in a list are in each row within a column

Right now I currently have a list of the most common words in column1: list1, that I've extracted from a csv file. How do I check each cell in column 2 to see if that cell contains any of the words in list1, and if so label it a 1 if it does have any of the words or 0 if it doesn't.
Sal Ha
0

votes
0

answer
5

Views

What's the best practice to scan a text file on linux for quality checking (pattern matching)?

We received data from clients for processing in Hadoop, first thing is the data will arrive as a file on linus in csv format. Usually the files are good, however, we've seen cases that there are a few lines with bad data - this caused the whole processing be rejected in the last audit phase. So it i...
mdivk
1

votes
1

answer
383

Views

pandas read .csv file without suitable delimiter. (only seperate first column vs “rest”)

I try importing a .csv file into python pandas as the following: dataframe = pd.read_csv(inputfile, sep=delimiter, header=None) However, each line of the (huge) inputfile consists of an integer, followed by some string. Like this: 1234 this string % might; contain 눈 anything The result should be a...
pcsso
1

votes
3

answer
143

Views

Combine three columns into one in CSV file with python and pandas

Hi I'm trying to combine several existing columns into 1 new column then delete the three original ones in a CSV file. I have been trying to do this with pandas however not having much luck. I'm fairly new to python. My code first combines several CSV files in the same directory an then attempts to...
Abob
1

votes
2

answer
43

Views

PHP: PDO + CSV export not downloading (headers issue?)

I'm having a hard time making my CSV export function work. I've found around plenty of examples using mysqli_* functions, but I'm actually using PDOs so I had to adapt some questions/answers to my needs. For what I can see, I'm properly parsing and writing data to the *.csv file, but at the end I do...
Davide3i
0

votes
0

answer
18

Views

Nested Json, Optimize with Python Recursion?

A huge thank you for those that helped with How to extract values from nested JSON array using pandas. My boss was happy with the results, and I was able to optimize the code slightly for grabbing more details: Policies- "description" and "groups" policy items - "users" and "database values" JSON FI...
Jaz
1

votes
2

answer
49

Views

Replacing and deleting columns from a csv using python

Here is a code that I am writing import csv import openpyxl def read_file(fn): rows = [] with open(fn) as f: reader = csv.reader(f, quotechar='"',delimiter=",") for row in reader: if row: rows.append(row) return rows replace = {x[0]:x[1:] for x in read_file("replace.csv")} dele...
Anuj Kulkarni
1

votes
3

answer
30

Views

Parsing text file with a complicated delimiter

I have a text file containing tweets in the following format ['RT @abcd: Name another swing group who can sell out Bob Field. Thank you.', 'Thanks to amazing supporters like @Bob Community Development, eligible New Yorkers can use NYC #FreeTaxPrep to get… SomeURL'] I want to convert the string to...
HMLDude
1

votes
3

answer
31

Views

Exporting Python Scraping Results to CSV

The code below is yielding a value for the "resultStats" ID, which I would like to save in a CSV file. Is there any smart way to have the " desired_google_queries" (i.e. the search terms) in column A and the "resultStats" values in column B of the CSV? I saw that there are a number of threads on thi...
M. S.
1

votes
2

answer
38

Views

Using csv.DictWriter to output an in-memory gzipped csv file?

I want to use a DictWriter from Python's csv module to generate a .csv file that's compressed using GZip. I need to do this all in-memory, so utilizing local files is out of the question. However, I'm having trouble dealing with each module's type requirements in Python 3. Assuming that I got the ge...
felipecgonc
0

votes
0

answer
5

Views

How to compare the particular columns of 2 CSV files through python

Python version - 2.6.6 1.csv 10.10.1.1,Web server,test,http,Running 10.10.1.1,Web server,test,http1,Running 2.csv 10.10.1.1,Web server,test,http,Not Running 10.10.1.1,Web server,test,http1,Not Running I have tried with below code to compare the 2 csv files for particular columns but it doesn't work....
Vijay
0

votes
0

answer
6

Views

How do I use the csv module and operator module together to sort CSV files

When I run this code it returns "index error list index out of range;" there are 13 items that are supposed to be in the list, and I have no idea why the list is returning a null field. I am unsure what to try. import csv import operator infile = open("datalist.csv", "r") csvfile = csv.reader(infile...
Hezekiah Bodine
0

votes
2

answer
31

Views

How to extract values from nested JSON array using pandas

I have a large JSON file (400k lines). I am trying to isolate the following: Policies- "description" policy items - "users" and "database values" JSON FILE - https://pastebin.com/hv8mLfgx Expected Output from Pandas: https://imgur.com/a/FVcNGsZ Everything after "Policy Items" is re-iterated the exac...
Jaz
0

votes
0

answer
3

Views

Executing Python Script in Django View [Upload CSV -> Batch Geocoding -> Show Link to Download by script generated CSV]

I found the following very useful script while Googleing: https://github.com/shanealynn/python_batch_geocode/blob/master/python_batch_geocoding.py This script is just what I need and it works flawlessly when I run it locally on my computer. I have a small Django site, where I would like to use this...
Tim
0

votes
0

answer
5

Views

using while loop in a csv file and getting values of a column (column having json data), but i got extra double quotes in each key value of column

getting extra doubles quotes in key value pairs while using while loop to get the column, values from a csv. structure of test.csv includes 1 column p, values of p = {"a"="1","b"={"h":"op" }} while IFS=, read p do echo ${p} done < test.csv >>output.txt Expected results = to get the column values a...
lucky
1

votes
2

answer
1.3k

Views

Execute stored procedure for each line in a CSV

I am trying to run a stored procedure using tsql multiple times in a query using each CSV line for the parameters. How would I go about looping through this? Cheers
Adam Howarth
0

votes
1

answer
17

Views

How to use if statements with pandas/ csv files

i want to check if the string data in a series is equal to a given string. but this returns: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all(). i know that to use and/or i use & / | but i don't understand how to do this with an if statement for i in range(...
BOBTHEBUILDER
0

votes
3

answer
30

Views

Python Script Unable to Get Cell Values

I am trying to accomplish a seemingly simple task in Python. This is now my third SO post in as many days and I am embarassed. I am hoping to open a csv file. I would like to loop through each row, and for every column in every row, if the value is NOT a 0 or a 1 (it is a "?"), to essentially rewri...
Jerry M.
1

votes
1

answer
6.3k

Views

mysql LOAD DATA INFILE not working

I am trying to use mysql LOAD DATA INFILE on command line to load csv file. It neither gives any error nor it loads any data. Can anyone please help me out? I am using following command mysql> LOAD DATA INFILE '/Applications/XAMPP/xamppfiles/htdocs/dev/__php__/mysql/data.csv' -> INTO TAB...
amique

View additional questions