Comment by Sundeep on Grep every other line of a file
You definitely should give sample input and output
View ArticleComment by Sundeep on Replacement of numbers between parentheses
Can you add the sed command you tried? Also, would the text to be extracted always start with an uppercase letter?
View ArticleComment by Sundeep on SED cmd to remove above line if particular value is...
@prasannakpechu what is your sed implementation? GNU/BSD/etc? Edit: your tag has linux/ubuntu and with GNU sed the command shown here shouldn't give an error
View ArticleComment by Sundeep on sed/grep to extract word between 2 characters
What is the expected output? lmn for both the given samples or just only for the first one or the second one? If you have GNU grep, does grep -oP '\^12=\K[^)^]*' solve your issue?
View ArticleComment by Sundeep on what is the fastest way to achieve this kind of alignment?
Use the column command
View ArticleComment by Sundeep on awk csv quoted field need to capitalize each word
Can be simplified to sed 's/\<\w/\u&/g' .... also, sed -E 's/\w+/\L\u&/g' if words can have mixed case (ex: ceO --> Ceo)
View ArticleComment by Sundeep on sed substring operations based on pattern match
There are numerous regex flavors. Sed typically supports BRE/ERE, which do not have features like lookarounds and do not support escapes like \d. Also, backreferences syntax is \N, so you'll need \1...
View ArticleComment by Sundeep on Remove multiple lines from a given match until an empty...
@rwst why do you need -r here? There are no ERE metacharacters used. (Also, -E is more portable over -r these days)
View ArticleComment by Sundeep on Copy first row to the last in file
Could also use 1t$ to copy the first line to the end (checked with GNU ed, not sure about other implementations).
View ArticleComment by Sundeep on Is there anything like the For Dummies books for...
There are learning resources in the tag info: stackoverflow.com/tags/awk/info. Also, check out ferd.ca/awk-in-20-minutes.html and backreference.org/2010/02/10/idiomatic-awk
View ArticleComment by Sundeep on find and grep - Suppress terminated by signal 13
Suggestion: use find -exec or find -print0 | xargs -0 as a good practice to avoid issues due to filenames.
View ArticleComment by Sundeep on Command-line pivot a long table into a wide table
If you are okay with building the header manually, try tail -n +2 ip.csv | datamash -t, -g1 collapse 3
View ArticleAnswer by Sundeep for Translate specific columns to rows
Here's a solution with newer versions of datamash (thanks glenn jackman for the tips about count and --collapse-delimiter):datamash -t'' -g1 --collapse-delimiter=';' count 2 collapse 2 sum 3...
View ArticleAnswer by Sundeep for How to grab every n,n+1 th lines from a text file
Here's one possible solution (guessing that this is the required output):$ seq 20 | sed -n '0~4{N;p}'458912131617As per the manual for the N command:Add a newline to the pattern space, then append the...
View ArticleAnswer by Sundeep for How to print lines between two patterns with optional...
One way is to reverse the input, get TAG2 to TAG1 and then reverse again:$ tac ip.txt | sed -n '/TAG2/,/TAG1/p' | tacTAG1some right textTAG2TAG1some right text 2TAG2Another way is to reset and start...
View ArticleAnswer by Sundeep for Trying to understand the behavior of grep - line seperator
Note: I'm not answering about -v and context matching, since I don't think they are well defined or even supposed to work. For example, this output doesn't make sense:$ seq 5 | grep -v -A1 '3'12345The...
View ArticleAnswer by Sundeep for Ignore line if any non-matching character is present
For the given sample input/output, try these:$ awk '!/[^a-z0-9]/' ip.txtabc123123abc$ grep -v '[^a-z0-9]' ip.txtabc123123abc[^set] means match any character excepts or e or t - in other words, ^ at the...
View ArticleAnswer by Sundeep for Accessing two fields of a line before a matched line
With GNU awk:$ awk '$1=="Special"{print m[1], m[2]} {match($0, /output1\s+(\S+).*output2\s+(\S+)/, m)}' ip.txt45 8038 99With perl:$ perl -ne 'print "@d\n" if /^Special/; @d = /output[12]\s+(\S+)/g'...
View ArticleAnswer by Sundeep for Fastest way to find lines from a large file in another...
Here's a solution using awk. Not sure if it will be faster than grep or ripgrep, but it is possible due to hash-based lookup. This assumes your RAM is big enough to load the first file (4.7 GB and 226...
View ArticleAnswer by Sundeep for splitting up big file using AWK, cannot get past 252...
You can also use csplit:csplit -qz ip.txt '/[0-9]\{2\}\/[0-9]\{2\}\/[0-9]\{4\}/''{*}'This will create files named xx00, xx01, xx02, etc. You can customize the output names. For example, -n1 -f'split'...
View ArticleAnswer by Sundeep for Linux search subdirectories and among .js .py files for...
You can use the --include option to filter files based on a glob pattern. For multiple globs, you can either use this option multiple times or use the brace expansion feature.echo --include={*.js,*.py}...
View ArticleAnswer by Sundeep for Move lines in file using awk/sed
Generic solution with GNU tac to reverse contents:$ tac -bs'>' ip.txt>ID.2GGAATACCACATCCCGCAGGGTTAAAAAAGAAAAAATCAGTAACAGTACTGGA>ID.1GGAACACGACATCCTGCAGGGTTAAAAAAGAAAAAATCAGTAAAAGTACTGGABy...
View ArticleAnswer by Sundeep for How to replace consecutive symbols using only one sed...
Lookarounds is helpful in such cases:$ s='t;2ABC;t;t;t;tortuga;fault;t;t;bored;t'$ echo "$s" | perl -lpe 's/(?<![^;])t(?![^;])/1/g'1;2ABC;1;1;1;tortuga;fault;1;1;bored;1
View ArticleAnswer by Sundeep for Using sed to add line above a set of lines
If input files aren't large to cause memory issues, you can slurp the entire file and then perform the substitution. For example:perl -0777 -pe 's/^O\nH\nH\n/FRAGNAME=H2ODFT\n$&/gm' ip.txtIf this...
View ArticleAnswer by Sundeep for How can I make my regular expression match variables...
If you are okay with installing ripgreprg -oU '\{[^}]*\} = process\.env'The -U option enables multiline matching.That said, grep -oz '{[^}]*} = process\.env' works for me for your given input samples....
View ArticleAnswer by Sundeep for How do I get rid of unwanted pipe symbols in a pipe...
Assuming you have fixed number of fields, here's one possible way to do it:perl -pe 's/^([^|]*\|){8}\K.*?(?=(\|[^|]*){6}$)/$&=~s,\|,\\|,gr/e' ip.txtThis will replace | with \| in the problematic...
View ArticleAnswer by Sundeep for Get last one-digit number after word
With GNU grep:$ echo 'Fund1 abcd2 Rail1 50 000 13.2 2Fund Train2' | grep -oP '[a-z]\K\d\b'1212-P option enables PCRE. [a-z]\K matches lowercase alphabets, but won't be part of the matched portion. \d...
View ArticleAnswer by Sundeep for How do I count multi line recurring pattern in a file?
With ripgrep:$ rg -UPc '(?s)^A$((?!^A$).)*?^XYZ$' ip.txt2The -U option enables multiline search. -P is for PCRE (since we need lookarounds) and -c is to get the count. (?s) enables . to match newline...
View ArticleAnswer by Sundeep for Combine two file according to a specific match
Here's one way to solve this:$ awk 'NR==FNR{a[$1]=$2 " " $5 " " $7; next} {print $0, $1 in a ? a[$1] : "NO MATCH"}' file2 file1ID1 -5.1 -4.4 juice ill,safe F25B9ID2 -6.2 -3.8 NO MATCHID3 -8.9 -7.1...
View ArticleAnswer by Sundeep for awk/sed: repeat 1st column for each column
Here's another awk solution:$ echo 'A B C D E F' | awk '{for(i=2; i<=NF; i++) $i=$i " " $1; sub(/^[^ ]+ /, "")} 1'B A C A D A E A F A
View ArticleAnswer by Sundeep for How to find last occurrence of pattern and print 20...
Since sample wasn't provided, I created one:$ cat ip.txtapple===123===bananafigmangocherry===423.14$ tac ip.txt | grep -m1 -A3 '===' | tacfigmangocherry===tac is handy for such cases. After reversing...
View ArticleAnswer by Sundeep for Need to get file name, line number, match position of a...
If you are okay with position starting with 1 (as opposed to 0), you can use ripgrep$ cat ip.txtabcdefxyzva$ rg -H --column --no-heading 'a' ip.txtip.txt:1:1:abcip.txt:3:5:xyzvaYou can use awk for a...
View ArticleAnswer by Sundeep for How to modify grepped output with cut or awk in Unix?
Based on given sample, here are solutions using sed and awk (they both support filtering, so no need grep):$ sed -nE '/Receivedrequest/s/[^[]+\[([^,]+),[^=]+=([^,]+),[^=]+=([^,]+),[^=]+=([^]]+).+/\1 \2...
View ArticleAnswer by Sundeep for linux command to transform 'VW_ABCD_EF_GHIJ_KLM_...'...
A slightly different solution, without the initial ${s,,} processing:$ s='VW_ABCD_EF_GHIJ_KLM'$ echo "$s" | sed -E 's/^VW//; s/_([^_]+)/\L\u\1/g'AbcdEfGhijKlm\L\u\1 will uppercase the first character...
View ArticleAnswer by Sundeep for Change random text between strings with sed
You need to use ERE for () to act as metacharacters.$ sed -E 's/(text3").*(" text4)/\1foo\2/g' ip.txttext1"text2" text3"foo" text4 text5"text6"text1"text2" text3"foo" text4 text5"text6"I'd recommend...
View ArticleAnswer by Sundeep for Group by a column and creat a new data frame using bash
With GNU datamash:$ <ip.txt datamash -t, -g1,2 min 3 max 4 first 5-6chr1,A,1,40,y,-chr2,B,0,30,y,-chr3,C,1,3,y,--t, use , as the field separator-g1,2 group by first and second columns (assuming...
View ArticleAnswer by Sundeep for how to print remaining portion of each line after a...
For a single character, you can use it as a field separator with cut and display except the first field. For example:$ echo 'log-2023-06-30-082821.txt:2023-06-30 08:30:32.648' | cut -d: -f2-2023-06-30...
View ArticleAnswer by Sundeep for regex in sed to keep current line but remove next line
Here's one way to do it (checked with GNU sed, syntax might vary for other implementations):$ sed -E '/Value[0-9]{1,2} [0-9a-z]{8}$/{n; d}' ip.txtServerAValue1 fh824rfzServerBValue3...
View ArticleAnswer by Sundeep for Get text between markers that span over multiple lines
OP's attempt can be simplified to delete the lines containing foo along with the start and end markers:sed -n '/START MARKER/,/END MARKER/ {//d; /foo/d; p}'This works with GNU sed, but other...
View ArticleAnswer by Sundeep for Sort output in bash based on character count not...
With perl:$ perl -e 'print sort {length($a =~ s/ //gr) <=> length($b =~ s/ //gr)} <>' ip.txtaa ba b c abcdeab cd efabcdefgWith ruby:$ ruby -e 'puts readlines.sort_by {_1.sub(" ", "").size}'...
View ArticleAnswer by Sundeep for accumulate multiple values against one record in awk
With newer versions of datamash:$ datamash -t'|' -c';' -g 1,2 collapse 3,5 <ip.txt1|dev|Smith|minus1|ana|jhon;peter|plus;plus2|dev|dash;|minus;plus-g 1,2 group by 1st and 2nd columnscollapse 3,5...
View Article
