Questions tagged [grep]

1

votes
2

answer
70

Views

Bash - More efficient way to process csv file than grep [closed]

Updated I have a file (file.txt) with a list of words: apple banana cherry orange pineapples I have a csv file (data.csv) that contains lots of data: 1,'tasty apples',3,5 23,'iphone app',5,12 1,'sour grapes',3,5 23,'banana apple smoothie',5,12 1,'cherries and orange shortage',3,5 23,'apple iphone or...
Lou Diamonds
1

votes
2

answer
51

Views

How to output first line of curl result in a file with bash script

I want to read a file of urls, curl each url and only get the first line which contains the HTTP code. I am running under Windows 10 inside Cmder. #!/bin/bash input='urls.csv' truncate -s 0 dest.csv while IFS= read -r var do result= `curl -I ${var%$'\r'} | grep HTTP $result` echo '$var $result' >> d...
Yvon Huynh
1

votes
1

answer
25

Views

Bash grep output in addition to echo within if statement

I have a simple script that checks if images are being used within our html directory. I have a file called imageserver.txt which looks like: imageserver/icons/socialmedia/sqcolor_tumblr.png imageserver/icons/socialmedia/sqcolor_gaf.png imageserver/icons/socialmedia/sqcolor_yelp.png imageserver/ico...
Zak
1

votes
2

answer
28

Views

How to grep strings that matches two (or more) variables in R

I have a list from which I would like to extract specific elements to put into a new list e.g. MyFullList = ( ' 'BLUEh473vs65hRED7vfd8edgf9GLOSS.csv', 'BLUE5654vsy56hgfREDjkgr7MATTE.csv', 'BLUEhjirhgccsREDgfigjINCAN.csv', 'BLUEvfdjhkldfgsGREENdfggh5GLOSS.csv', 'BLUEtgergvsGREENsghjkhgnMATTE.csv'...
Solebay Sharp
1

votes
3

answer
52

Views

need to clean file via SED or GREP

I have these files NotRequired.txt (having lines which need to be remove) Need2CleanSED.txt (big file , need to clean) Need2CleanGRP.txt (big file , need to clean) content: more NotRequired.txt [abc-xyz_pqr-pe2_123] [lon-abc-tkt_1202] [wat-7600-1_414] [indo-pak_isu-5_761] I am reading above fil...
Junipar70
0

votes
2

answer
15

Views

Case insensitive search between two strings in file

I have a file constructed the following way: # comment with keyword One some code here and some more code # another comment with keyword Two some code # more comments and keyword Three lots of code I'm trying to extract the comments and corresponding code lines given a keyword, in a case-insensitive...
marshmcintire
1

votes
3

answer
51

Views

Copy numbers at the beginning of each line to the end of line

I have a file that produces this kind of lines . I wanna edit these lines and put them in passageiros.txt a82411:x:1015:1006:Adriana Morais,,,:/home/a82411:/bin/bash a60395:x:1016:1006:Afonso Pichel,,,:/home/a60395:/bin/bash a82420:x:1017:1006:Afonso Alves,,,:/home/a82420:/bin/bash a69225:x:1018:100...
Tigaz Marques
1

votes
2

answer
66

Views

How to search and replace with egrep and sed on macOS?

I want to match a pattern in a file and replace it. This command works with egrep, xargs and sed: egrep -lRZ 'hello' . | xargs -0 -l sed -i -e 's/hello/world/g' The problem: It does not work on MacOS because the xargs of MacOS does not support the argumente -l. xargs: illegal option -- l usage: xarg...
Alpha
1

votes
3

answer
70

Views

How to extract email headers extending on multiple lines from file

I am trying to extract the To header from an email file using sed on linux. The problem is that the To header could be on multiple lines. e.g: To: [email protected], [email protected], [email protected], [email protected], [email protected] Message-ID: I tried the following: sed -n -e '/^[Tt]...
Anis Bedhiafi
1

votes
2

answer
70

Views

need help on shell script for expected output

I have an input file called input.txt like this: powerOf|creating new file|failure creatEd|new file creating|failure powerAp|powerof server|failureof file I extract the text up to just before the fist capital letter in the first field and store those snippets in output.txt: power creat I used the...
Mohan
1

votes
3

answer
80

Views

From grep to perl: inverse matching by perl

it is pretty long time that I don't program in perl and I need some push. I need to compare two lists in order to retain only the lines that are not matching. my first file looks like this: 1 pf1 er2 0,4 2 pf1 er3 0,56 3 pf1 er6 0,72365 4 er3 pf3 0,263 5 pf5 er2 0,28473 The second fi...
Gabelins
1

votes
6

answer
105

Views

Script using sed and grep gives unintended output

I have a 'source.fasta' file that contains information in the following format: >TRINITY_DN80_c0_g1_i1 len=723 path=[700:0-350 1417:351-368 1045:369-722] [-1, 700, 1417, 1045, -2] CGTGGATAACACATAAGTCACTGTAATTTAAAAACTGTAGGACTTAGATCTCCTTTCTATATTTTTCTGATAACATATGGAACCCTGCCGATCATCCGATTTGTAATATACTTAACTGCT...
S.Chereddy
1

votes
5

answer
50

Views

UNIX average of specific employee as per designation

This is an example of a text file to be given as input Name,Designation,Salary Hari,Engineer,35000 Suresh,Consultant,80000 Umesh,Engineer,45500 Maya,Analyst,50000 Guru,Consultant,100000 Sushma,Engineer,30000 Mohan,Engineer,30000 My code should be able to run find the average salary of particular emp...
John Doe
1

votes
3

answer
117

Views

How to delete content between text?

I want to delete all in all html files in all subfolders. I can't find the correct version of the line regular expression:
Canapsis
1

votes
7

answer
5k

Views

Find lines containing all keywords in bash script

Essentially, I would like something that behaves similarly to: cat file | grep -i keyword1 | grep -i keyword2 | grep -i keyword3 How can I do this with a bash script that takes a variable-length list of keyword arguments? The script should do a case-insensitive match of lines containing all keywords...
Siou
0

votes
3

answer
36

Views

How to clean/format awk output?

Trying to grab just the latest version number which will always be behind 'misc' from ftp://ftp.adobe.com/pub/adobe/reader/mac/AcrobatDC/ I might be able to fix this by using 'cut' but it might not stick with the next latest version if the time is x:xx. Current code: latestver=`/usr/bin/curl -s -L...
DankDalf
1

votes
2

answer
798

Views

Don't understand CIRCUMFLEX use in this UNIX command ls -L | grep -v ^zip

ls -L | grep -v ^zip What i understand from this command is, it is doing a listing and piping the output to a grep command which performs grep on the received output file list to select any file with file name not containing text 'zip' and any file with file name containing text 'zip'. If this is co...
Oh Chin Boon
0

votes
2

answer
16

Views

Need help matching a mattern using grep/egrep in bash scripting

I am trying to match all characters of given string but those characters should match in the order as given to the bash script. while [[ $# -gt 0 ]]; do case $1 in -i) arg=$2 egrep '*[$arg]*' words.txt shift ;; esac shift done $ sh match_the_pattern.sh -i aei words.txt Should return words like abst...
Bhavika
1

votes
1

answer
59

Views

string extraction and dupes filtering mac OS X

I have a bunch of files with sql log. I'm looking out to extract all occurrences of the following pattern SQL log has sql that looks something like this sel * from DB.T1; update DB.T1; delete from DB.T2; collect stats on DB.T3 index (a,b,c); sel count(*) from Db.T1; sel count(*) from db . T2; sel...
pmv
1

votes
0

answer
232

Views

grep command in json file

I am using grep command in a multiple json file which has content like this: 'Verdict': { 'Severity': '-1', 'Description': 'Some description' Reputation' } I want look for word Verdict and if it matches ,get the Severity number. So far I used: 'grep -A 2 -R \'Verdict\' path\to\file*.json\'|grep Se...
Piku
1

votes
2

answer
31

Views

How to NOT include certain no. of patterns in the output?

I have a command - some command | egrep -i '($query_value)' which returns lines like - > 2018-01-12 23:17:05.097 Info: Deleted 6 MB at 1129 MB/sec > /Meters/00002c97 2018-01-12 23:17:05.104 Info: Deleted 6 MB at 864 > MB/sec /Meters/00002c99 2018-01-12 23:26:00.751 Info: Saving > /Meters/00002c9...
bosari
1

votes
3

answer
62

Views

replace variables of a file with reference to values in another file

I have two files file1.txt variable1=string1 with spaces characters variable2=string2 with spaces characters variable3=string3 with spaces characters file2.txt sometext1 textvariable1 sometext2 sometext3 variable2 sometext4 sometextcharactersvariable1 charactersvariable3 sometext5 variable2 ...........
venkatsai
1

votes
1

answer
39

Views

how to capture one more column values in shell

HI I am new to shell script.I have a data like this. 2018-01-18T16:23:25,389 INFO [HiveServer2-Background-Pool: Thread-63([])]: ql.Driver (Driver.java:execute(1735)) - Executing command(queryId=hive_20180118162325_5ad8be3f-80e7-468d-bb47-1bdc2d2fb624): select * from aa limit 5 2018-01-18T16:23:2...
Arun Kumar
1

votes
2

answer
58

Views

Regular expressions to remove content in a JSON file. using BBEdit

I am trying to get rid of a load of content in a JSON file to have it prepped for translation using BBEdit. a Line looks like this: 'ribbonText1' : 'The text that needs to be translated', I would like to remove everything so it ends up like this: The text that needs to be translated Any help greatly...
user9305836
1

votes
2

answer
50

Views

Unix grep multiple character search

I'm working on a problem in unix bash where it wants me to get an input from the user and then find if the input is valid (i.e. contains only A-Z a-z and the _ underscore character) and reject anything with a character other than those. For example Space_oddity would be valid while Space_oddity132 w...
masral
1

votes
3

answer
91

Views

Optimal way to recursively find files that match one or more patterns

I have to optimize a shell script, but after one week, i didn't succeed to optimize it enough. I have to search recursively for .c .h and .cpp file in a directory, and check if word like this exist: 'float short unsigned continue for signed void default goto sizeof volatile do if static while' words...
HectoFR
1

votes
2

answer
87

Views

unix combine grep w and v command

I want to search a file and include the text #!/bin/bash, but exclude any other line that has a # sign. These two commands: grep -w '#!/bin/bash' file and grep -v '^#' file each do one part of this job. I would like this to be a single command, so here's what I've tried. grep -w '#!/bin/bash' | gre...
Some_Dude
1

votes
2

answer
79

Views

How can I find both identical and similar strings in a particular field in a text file in Linux?

My apologies ahead of time - I'm not sure that there is an answer for this one using only Linux command-line fu. Please note I am not a programmer, but I have been playing around with bash and python a bit over the last few years. I have a large text file with rows and columns that resemble the fol...
logoi5893
1

votes
1

answer
30

Views

removing lines that don't match by grep

I have a file containing gene names of interest (24423 genes), and another file containing the lengths to all the genes (41306 genes). I want the lengths only to the 24424 genes, but when I grep using grep -wf file1 file2 or even fgrep -Fwf file1 file2, I get some excess genes, because some genes in...
vinay rao
1

votes
1

answer
26

Views

what characters are matched by '\w' in grep search?

Is there any way to know with certainty whether a particular Unicode point is matched or isn't by '\w' (apart from running searches on all the characters I want to know about)? It's straightforward in English but less so in a language like Khmer, which has characters used in words that apparently do...
viverechristus
1

votes
1

answer
68

Views

gawk grep piping redirection to output file not working

I have this file.txt: a b c a f g e h j I wrote an awk script that does BEGIN {...} {...} END { a = 'a' b = 'b' system('grep ' a ' file.txt | grep ' b ' > t') } I expect it to print a b c in file t. Running the same script from ConEmu on Windows 7 will produce an empty t file. On the other hand, exe...
Daemon Painter
1

votes
3

answer
45

Views

grep for contents after pattern for word character and comma

echo 'this is a test:foo,bar,baz']' | grep -o -E 'test:.*' | awk -F: '{ print $2 }' foo,bar,baz'] I get '] printed at the end, how to print only the word characters and common, nothing else, in this case I need to extract only foo,bar,baz
rodee
1

votes
3

answer
135

Views

How to parse xml file with perl based on attribute using grep

I'm new to perl and have been struggling. I have an xml file with the following structure, but with thousands of entries: test.xml I need to parse/delete the 'spectrum_query' nodes that do not contain in the attribute 'spectrum' the string in this example 'TPP08' i.e. in reality what it is between t...
Sasha
1

votes
4

answer
52

Views

Extract text between 2 markers

input: random adsf 845 asdsf.$Ecdda-string0.rand me39 84abd dee rand a[s% 845 a1sEdsf.$cdNda-string1.rand me39 84abd output - keep everything between da- and . string0 string1 what I have tried so far: sed -e 's/.*da-\(.*\)./\1/' file grep -o -P '(?
achille
1

votes
2

answer
127

Views

awk sed grep to extract patten with special characters

I am trying to understant the switchs and args in awk and sed For instance, to get the number next to nonce in the line form the file response.xml: WWW-Authenticate: ServiceAuth realm='WinREST', nonce='1828HvF7EfPnRtzSs/h10Q==' I use by suggestion of another member nonce=$(sed -nE 's/.*nonce='([^']+...
Pedro Caldeira
1

votes
5

answer
78

Views

I am trying to replace a text for example

Example: 'word' -nothing To word' - nothing in gvim. I tried :%s/^.*\'/ But what I get is: -nothing Well I am new to scripting so I would like to know if it can be done in any other way like using gvim or awk or sed.
Vinut V Patil
1

votes
1

answer
294

Views

How to grep umlauts and other accented text characters via AppleScript

I have a problem trying to execute shell scripts from apple script. I do a 'grep', but as soon as it contains special characters it doesn't work as intended. (The script reads a list list ob subfolders in a directory and checks if any of the subfolders appear in a file.) Here is my script: set searc...
Sebastian Schöps
1

votes
0

answer
36

Views

grep or detect a pattern in a character that begins with a phrase?

Using Rstudio. Have a descriptive character feature that begins with values like 'I love you', 'I love him', 'I love my dad', 'I rather love...', 'I hate..', 'I don't care..', 'I surely love....'. Many 'I * love' patterns, among others. Now I like to create a new feature that =1 if the raw feature...
opt135
1

votes
1

answer
57

Views

Grep finds the string although the string is not visible in file

I am stuck since 1day with a weird problem. I have a CSV file which I need to import into my hive table. The CSV file, however, has newline characters embedded in between the strings. As the files are huge I am not able to use a text editor to replace the '\n' character. I wrote a python program to...
Sridhar Iyer
1

votes
1

answer
186

Views

ssh remote command with wildcard

My goal is to get all IPs logging into my web server by checking access log files. nginx log files look like this: ls -la /var/log/nginx/access.log* /var/log/nginx/access.log /var/log/nginx/access.log1 /var/log/nginx/access.log2 ... ... running this locally works: sudo grep loging /var/log/nginx/acc...
jinzoo

View additional questions