Questions tagged [awk]

11025 questions
0

votes
0

answer
4

Views

Transpose columns of data table using awk

This is a follow thru with regards to my previous question: (Pivot table content using awk command), i wanted to transpose the data. see the expected output in below code(from oguz): awk 'BEGIN {FS=OFS=SUBSEP=','} NR>1{a[$2,$4,$5];b[$2,$4,$5,$1]=$3} END{print 'TESTER','PROGRAM','AREA','High','Low','...
impitnin
1

votes
5

answer
88

Views

Match strings from two files and output fields in specific order

I have the following two files. query.txt pumpkin kiwi subject.tsv kiwifruit something green melon something red pumpkinhead something orange I'd like to loop through query.txt and check for each line if any first field in subject.tsv contains the line. If yes, a tab plus the third field f...
Matthias
1

votes
2

answer
30

Views

Sort numeric values except text in Bash

I have a csv file called prices.csv that looks like this: Name, Price, Description Apple, 2.85, fruit Kiwi, 1.96, fruit Banana, 0.99, fruit Peach, Not available, fruit Orange, 2.02, fruit I would like to sort the second column (Price) in ascending order, except for the values that are 'Not available...
vincers
1

votes
2

answer
55

Views

parse file and date manipulation in bash with a large file

I'm looking for a better way to manipulate a date format into something that I want. I do manage to do it, but I have to process the files several times because I can not get date to do it in one pass. The format I have: Wed Jan 30 08:00:00 2019 : misc data The format I want: 30/01/2019 08:00:00 : m...
Madivad
1

votes
2

answer
50

Views

Merge File1 with File2 (keep appending from File1 to File2 until no more rows)

I can't find a solution. So here is the problem. Result should be 100 rows (File1) with contents from File2 repeating 25 times. What I want is to join the contents even though the number of rows is not equal. Keep repeating including lines from File2 until number of rows from File1 is met. File1: te...
Jax
1

votes
2

answer
43

Views

awk code to output lines with matching fields, but no output

I'm trying to output lines in one file fileB that are matched with parameters in another file fileA, but my attempt doesn't output anything. fileA: chr1 7237 7238 chr1 7637 7637 chr1 7908 7908 chr1 8045 8045 chr1 8329 8329 chr1 8358 8358 chr1 8375 8375 chr1...
DangIt
1

votes
1

answer
69

Views

How does gawk -e 'BEGIN {' -e 'print “hello” }' work?

Gawk 5.0.0 was released on April 12, 2019. Going through the announcement I found this: Changes from 4.2.1 to 5.0.0 (...) 11. Namespaces have been implemented! See the manual. One consequence of this is that files included with -i, read with -f, and command line program segments must all be self-co...
fedorqui
1

votes
1

answer
44

Views

Why does storing my regex in a variable break the comparison test? (Awk/Gawk)

I have a file called regeces containing a number of line-by-line regexes. Here's an example: The .* is .* years old I like .* with lots of .* I also have a data file very creatively called data: The car is 3 years old I like pizza with lots of pepperoni I like tacos with lots of cheese The house is...
Andrew
1

votes
1

answer
20

Views

convert first column in a csv file from timestamp to year-month format

Trying to convert first column in a csv file from unix timestamp to date(year-month format) Tried date -d @number'+%Y-%m' and awk, but awk doesn't recognize @ when used together Extract from a csv file : 1556113878,60662402644292 1554090396,59547403093308 Expected O/p 2019-04,60662402644292 2019-03,...
Bt_code
1

votes
2

answer
32

Views

how to insert line break after a specific string?

I have a file with the following text, I want to make a line break, after the S. character i have this 1900-01-01 00:00:00|1|S|S|S|S.1900-01-01|S.1900-01-01 00:00:00|1|S|S|S|S.1900-01-01 00:00:00|1|S|S|S|S. i want 1900-01-01 00:00:00|1|S|S|S|S. 1900-01-01|S. 1900-01-01 00:00:00|1|S|S|S|S. 1900-01...
Brayan Pastor
0

votes
2

answer
19

Views

Manipulate AWK field variables ($1,$2,..) and add that to the end of line

I am trying to create /etc/host file from ansible inventory file. Ansible inventory file looks like this: [masters] k8s-master-a.example.com ansible_ssh_host=10.15.90.2 k8s-master-b.example.com ansible_ssh_host=10.15.90.3 k8s-master-c.example.com ansible_ssh_host=10.15.90.4 [etcd] k8s-etcd-a.examp...
bujail
0

votes
5

answer
97

Views

Sum column and count lines

I am trying to sum certain numbers in colum 2, it works with my code. But I want to count also how many times the same value in colum 2 is repeated and print in the last column. file1 36 2605 1 2 36 2605 1 2 36 2603 1 2 36 2605 1 2 36 2605 1 2 36 2605 1 2 36 2606 1 2 Output Desired 2603 36 1...
OXXO
1

votes
7

answer
93

Views

Replacing characters in each line on a file in linux

I have a file with different word in each line. My goal is to replace the first character to a capital letter and replace the 3rd character to '#'. For example:football will be exchanged to -> Foo#ball I tried thinking about using awk and sed.It didn't help me since (to my knowledge) sed needs an ex...
Ben Zoker
-1

votes
0

answer
16

Views

Making a new column out of specific lines from within a file?

I am trying to solve the following problem but several attempts have failed to get it right. Supposing I have the following in a file: a,b,c d,e,f g h I need some way to add g and h as a third column so that the file looks like this: a, b, c, g d, e, f, h Thanks for you help.
John Davy
1

votes
3

answer
1k

Views

AWK - match multiple patterns and print results in one line

Using my limited scripting knowledge.. I have put together an expect script to run some commands on specific devices. The output is below and is saved from all devices to a file (I just listed 2 devices as an example). Current State : Active Router1slot1# showstats Active stats : 31 Ac...
dxx bstu
0

votes
0

answer
6

Views

Sum of 2nd and 3rd column for same value in 1st column

I want to sum the value in column 2nd and 3rd column for same value in 1st column 1555971000 6 1 1555971000 0 2 1555971300 2 0 1555971300 3 0 Output would be like 1555971000 6 3 1555971300 5 0 I have tried below command awk -F' ' '{b[$2]+=$1} END { for (i in b) { print b[i],i } } ' but this seem...
Prince Garg
0

votes
2

answer
15

Views

Case insensitive search between two strings in file

I have a file constructed the following way: # comment with keyword One some code here and some more code # another comment with keyword Two some code # more comments and keyword Three lots of code I'm trying to extract the comments and corresponding code lines given a keyword, in a case-insensitive...
marshmcintire
1

votes
3

answer
51

Views

Copy numbers at the beginning of each line to the end of line

I have a file that produces this kind of lines . I wanna edit these lines and put them in passageiros.txt a82411:x:1015:1006:Adriana Morais,,,:/home/a82411:/bin/bash a60395:x:1016:1006:Afonso Pichel,,,:/home/a60395:/bin/bash a82420:x:1017:1006:Afonso Alves,,,:/home/a82420:/bin/bash a69225:x:1018:100...
Tigaz Marques
1

votes
4

answer
77

Views

Get only third and sixth column from command output of pdffonts

I am using poppler pdffonts to get fonts in a pdf document. Below is the sample output $ pdffonts 'some.pdf' name type encoding emb sub uni object ID ------------------------------------ ----------------- ---------------- --- --- --- --------- Tim...
Satheesh K
1

votes
2

answer
47

Views

Splitting the first column of a file in multiple columns using AWK

File looks like this, but with millions of lines (TAB separated): 1_number_column_ranking_+ 100 200 Target 'Hello' I want to split the first column by the _ so it becomes: 1 number column ranking + 100 200 Target 'Hello' This is the code I have been trying: awk -F'\t' '{n=split($1,a,'_');for (...
Steveman30290
1

votes
3

answer
54

Views

Something is wrong by compilation

What do I wrong please? I have a code in file file.awk: BEGIN { CONVFMT='%0.17f' } /D Format/ { in_f_format=0; next } /F Format/ { in_f_format=1; next } in_f_format != 1 { next } !($1 ~ /^[1-9]/) { next } $1 == 11 { prt(180,3.141592653589); next } $1 == 15 { prt(100,1); next } $1 == 20 { prt(10,1);...
Elisabeth
1

votes
5

answer
67

Views

Grabbing a word and variable number of characters afterwards

I have a TAB separated file containing long strings, which I intend to filter the 8th column which has a mix of letters and characters randomly, to containing only ';OCCURRENCE=some words|' part as below: input.txt 1 138440 CAMEL C T . . MANY-WORDS-AND-CHARACTERS(*/-;|\);OCCURRENCE=HOY-BR|MANY-WORDS...
bapors
1

votes
3

answer
70

Views

How to extract email headers extending on multiple lines from file

I am trying to extract the To header from an email file using sed on linux. The problem is that the To header could be on multiple lines. e.g: To: [email protected], [email protected], [email protected], [email protected], [email protected] Message-ID: I tried the following: sed -n -e '/^[Tt]...
Anis Bedhiafi
1

votes
5

answer
82

Views

sort text file by groups of 2 lines

I have a file with the following structure: A 35 74 dsadasd/1 0 + A 95 74 dsadasd/2 0 - B 75 159 dsadasd/2 0 + B 78 852 dsadasd/1 0 - C 12 789 dsadasd/1 0 + C 91 546 dsadasd/2 0 - A 87 52 dsadasd/2 0 + A 52 15 dsadasd/1 0 - I would like to...
Angel Garcia Campos
1

votes
5

answer
65

Views

Split 2nd occurrence of pattern of camel style text in sed

I am trying to create a key, value strings table for a mac app using sed and awk. So far I have got it to the point of having lines like: 'exif:DateTimeOriginal' = 'DateTimeOriginal:\t'; I want to do a final step to get: 'exif:DateTimeOriginal' = 'Date Time Original:\t'; In other words split up the...
Jim Merkel
1

votes
3

answer
43

Views

how to remove lines based on a number

I have a data like this >sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1 305 >sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2 534 >sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=...
Learner
1

votes
6

answer
105

Views

Script using sed and grep gives unintended output

I have a 'source.fasta' file that contains information in the following format: >TRINITY_DN80_c0_g1_i1 len=723 path=[700:0-350 1417:351-368 1045:369-722] [-1, 700, 1417, 1045, -2] CGTGGATAACACATAAGTCACTGTAATTTAAAAACTGTAGGACTTAGATCTCCTTTCTATATTTTTCTGATAACATATGGAACCCTGCCGATCATCCGATTTGTAATATACTTAACTGCT...
S.Chereddy
1

votes
3

answer
56

Views

Select current and previous line if values are the same in 2 columns

Check values in columns 2 and 3, if the values are the same in the previous line and current line( example lines 2-3 and 6-7), then print the lines separated as , Input file 1 1 2 35 1 2 3 4 50 1 2 3 4 75 1 4 7 7 85 1 5 8 6 100 1 8 6 9 125 1 4 6 9 200 1 5 3 2 156...
OXXO
1

votes
3

answer
54

Views

How to remove space after the first pattern in sed

I have a file file.txt Name-6.1.4_Float Time;453802;464204;466345 464481 466131 464283 465065 462561 Name-6.1.4_Logic Good Time;125896;145349;137303 131601 144520 132487 133884 I want to replace the spaces between the numerical value by ; after the first semicolon ; The spaces between the numeri...
Arya
1

votes
2

answer
67

Views

Match value in two files and replace values in selected columns

The purpose is to check if values for column 3 and 4 in file1 match with column 1 in file2. If any value match, then replace values in file2 for column 2 and 3 using the information of file1 columns 5 and 6 Also, I need add the values of columns 7 and 8 from file1 into file2 in columns 1 and 2 for t...
OXXO
1

votes
2

answer
38

Views

AWK ternary operator conditional statement not working

I have code that checks the contents of a variable that is created from parsing a csv file. however the code below isn't working. $3 ~ ($2 == '\'[ABCDEFGUHIJKLMNOPQRSTUVWXYZ]\'' ? '^\'[[:digit:]]\'$' : '\'\'$') { print '15th field invalid-OFFENCE FILE' } sample data below 'ABC','A','','a' --- # This...
jordanb111
1

votes
2

answer
36

Views

Filtering Data in CSV file

I have CSV file in this format a,b,c,d,e,f,no disk detected a,b,c,d,e,f,disk run into error a,b,c,d,e,f,no memory in the server a,b,c,d,e,f,memory has correctable errors in the last column, I need to search for the word disk and replace it with disk error memory and replace it with a memory error T...
Prashant Lakhera
1

votes
3

answer
66

Views

Convert a key:value file w/ comments into JSON document with UNIX tools

I have a file in a subset of YAML with data such as the below: # This is a comment # This is another comment spark:spark.ui.enabled: 'false' spark:spark.sql.adaptive.enabled: 'true' yarn:yarn.nodemanager.log.retain-seconds: '259200' I need to convert that into a JSON document looking like this (note...
jamiet
1

votes
5

answer
50

Views

UNIX average of specific employee as per designation

This is an example of a text file to be given as input Name,Designation,Salary Hari,Engineer,35000 Suresh,Consultant,80000 Umesh,Engineer,45500 Maya,Analyst,50000 Guru,Consultant,100000 Sushma,Engineer,30000 Mohan,Engineer,30000 My code should be able to run find the average salary of particular emp...
John Doe
1

votes
2

answer
113

Views

Optimized way to calculate time duration in BASH

I am using below function that I wrote in BASH to calculate time duration and decide if the duration is more than 5 minutes it returns yes,otherwise no. It is working as expected but the files I am processing are huge (millions of lines) and it takes very long time to finish the job. I am looking fo...
Ibraheem
1

votes
2

answer
85

Views

Need first two characters from a file and make sure that they both aren't below 6 then return line

I have a text file like this 17 Blue 45 Purple And I need to make sure the first two digits aren't both less than the number 6 and then print them. So, for example, the first line would print since the first digit 1 is lower than 6 and the second digit 7 is higher than 6, so they aren't both lower t...
serenitynow13
1

votes
1

answer
49

Views

More input files in awk

I have a code: /Curve No./ { in_f_format=1; next } /^[[:space:]]*$/ { in_f_format=0; next } {sum2+=$2; sum3+=$3} END{printf('%.6f\n',sum3/sum2)} That find a table in text file and provide a computation. The result is one number from one file. How to get a column of results for more files. I wrote:...
Lukáš Altman
1

votes
3

answer
132

Views

Is it possible to pass a script to awk inside a shell variable?

Is it possible to store an awk script inside a shell variable; for example: export script=''{printf(\$2); printf('\'\\n\'');}'' echo $script '{printf($2); printf('\n');}' The script functions properly when I call it directly as such: awk '{printf($2); printf('\n');}' testFile.txt prints proper outpu...
user3672527
1

votes
6

answer
73

Views

How can I print only lines that are immediately preceeded by an empty line in a file using sed?

I have a text file with the following structure: bla1 bla2 bla3 bla4 bla5 So you can see that some lines of text are preceeded by an empty line. I understand that sed has the concept of two buffers, a pattern space buffer and a hold space buffer, so I'm guessing these need to come in to play here, b...
user2061575
0

votes
3

answer
36

Views

How to clean/format awk output?

Trying to grab just the latest version number which will always be behind 'misc' from ftp://ftp.adobe.com/pub/adobe/reader/mac/AcrobatDC/ I might be able to fix this by using 'cut' but it might not stick with the next latest version if the time is x:xx. Current code: latestver=`/usr/bin/curl -s -L...
DankDalf

View additional questions