Problem using AWK in shell script

[kubicle]

For portability, you should probably use /bin/bash in the shebang (#!/bin/bash) instead of /bin/sh.
Depending on the system you run the script on, "/bin/sh" could the original Bourne Shell or a symlink to bash, ash or dash (like it is on *buntus).

Not necessarily the culprit here, but it can lead to some inconsistencies in syntax and execution.

[MoonRise]

Tried at of curiosity and I still get a blank file.

So, it works explicitly typed in konsole on my machine (moonrise@Bobcat:~/EDI867$ awk -F $'\t' '{$6 != ""}' Input.txt >> Output.txt). It works in cygwin64 environment in the shell script as seen in post #24 here. However that same script does not work when run from konsole here on my machine. Really baffled by that. It's something simple I'm sure. :/

[Snowhog]

Tried at of curiosity and I still get a blank file.

So, it works explicitly typed in konsole on my machine (moonrise@Bobcat:~/EDI867$ awk -F $'\t' '{$6 != ""}' Input.txt >> Output.txt). It works in cygwin64 environment in the shell script as seen in post #24 here. However that same script does not work when run from konsole here on my machine. Really baffled by that. It's something simple I'm sure. :/

"...does not work when run from konsole here on my machine." What machine would that be, your gigabyte Desktop or MSI Laptop, or some other? What version of bash (or as claydoh suggested, another?)?

[Snowhog]

Tried at of curiosity and I still get a blank file.

So, it works explicitly typed in konsole on my machine (moonrise@Bobcat:~/EDI867$ awk -F $'\t' '{$6 != ""}' Input.txt >> Output.txt).

That awk statement is not what I provided back in the modified script that does work with bash as tested on my iMac.

Yours: awk -F $'\t' '{$6 != ""}' Input.txt >> Output.txt
Mine : awk -F $'\t' '$6 != ""' $myFl >> $myFlf

Remove the braces {}

[MoonRise]

Snowhog;

Sorry to confuse on that. Yes, that is correct. awk -F $'\t' '$6 != ""' $myFl >> $myFlf is the one that worked on the cygwin64 setup as a shell script. The other does not. Then again, that doesn't work on my machine in a shell script and that is what is confounding.

Yes, it is on the MSI, but the Gigabyte did the same. Both are K18.04. Konsole version 17.12.3.

[Snowhog]

Going on a 54-mile bike ride at 9:30 this morning. I'll check the script on my 18.04 install when I get back this afternoon or early this evening. As I said, your script, as modified, does work using the terminal (and running the script with bash) on my iMac. I'll follow-up how it fares on my 18.04.

[MoonRise]

Enjoy the bike ride!

Yes, that modified script does work on the cygwin64 environment. Works great. Have it running now which is where I needed it to run. Just can't get it running on the 18.04 side of things where I tend to develop my programs.

[Snowhog]

So to satisfy curiosity, is the cygwin64 environment using bash or the Borne Shell? And if bash, which version?

[MoonRise]

Hope this answers the question.

[andystmartin]

I think I see the solution to this riddle. It's a combination of the script starting with

#! /bin/sh

and the

-F $'\t'

construct with your awk command line arguments.

Bash interprets $'xxx' to expand backslash-escaped sequences to their ANSI equivalent. In other words,

-F $'\t'

is an attempt to tell awk that the FieldSeparator is the TAB-character.

However, your script starts with

#! /bin/sh

which says interpret this script with /bin/sh (not bash).

If /bin/sh is dash or some other non-bash shell, $'xxx' expansion does not work.

Change the first line of your script to

#! /bin/bash

and the behavior of your script should match your command line.

[Snowhog]

I've just verified that the modified script I provided works perfectly in my Kubuntu 18.04 installation, and, it works with either the shebang of #! /bin/sh or #! /bin/bash, meaning that at here, sh is pointing to bash. That the script doesn't work on your Kubuntu 18.04 with #! /bin/sh tells me that you in fact, have bourne shell installed in addition to bash, and that /bin/sh isn't a link to /bin/bash, but actually points to/executes bourne shell. So, change the shebang in your script (on your Kubuntu 18.04) to #! /bin/bash and try the script again. Bet it works.

[MoonRise]

So I had done that with the Shebang BUT with the awk command I used at the command line out right. Didn't work. HOWEVER, with the modified awk command you gave me that worked on cygwin it actually did work this time here. Simple but that wasn't an easy find, at least I never would have.

Not sure what my system is pointing at really as it is really a base install. I've not personally chosen one shell over another.

THANK YOU!!!!

[MoonRise]

Happy Thanksgiving!!!!

[Snowhog]

So I had done that with the Shebang BUT with the awk command I used at the command line out right. Didn't work. HOWEVER, with the modified awk command you gave me that worked on cygwin it actually did work this time here. Simple but that wasn't an easy find, at least I never would have.

Not sure what my system is pointing at really as it is really a base install. I've not personally chosen one shell over another.

THANK YOU!!!!

Happy to have helped you get this worked out. I have to give nearly all the credit to my brother-in-law; he knows way more about this stuff than I do!