10 must know find and replace command in UNIX

 

In Unix often we are asked to perform certain things which requires us to find a particular pattern and then to play or manipulate that particular pattern, so now today let us learn how to be a master in that. The list here present 10 must know find and replace command in Unix with explanation.

1. Finding any extension files:

Our first example is how to find a set of files and then play with the files, to play we mean to manipulate the files. The first example deals with finding files which are older than certain time or heavier than certain, we do this most often with log files. Now lets find files which are 15 days old.

We can find files with any extension with *.* feature and then we can find with respect to what option we use with it, for example we will use the mtime option here, mtime is for modification time.

find *.* -mtime -15

find does the recursive search as well via find / -name file name we know that / searches from the root.

2. Finding particular set of files:

find . -name “*.c” -print.

. (dot) searches in current directory and will give relative path.*.c will give only the c extension files. Finding files which are one year old will be found by the below command, so the files which are not accessed from a year will be as below.

3. Finding old files with access time:

Find $HOME will search in the home directory

find /home -atime +365 -print

4. Finding and  negating:

Now the below command will select everything but the c files, so this option negate what we specify and find everything other than that.

find .! -name “*.c” -print

5. Other options in Find:

Now we can also use options with the find command. For example the size option will find the files in order of its size, the inum with the inode number details and type f with finding only the files. Example :

-size +2048 ;    -inum 12345 ;    -type f.

6. Deleting old files:

Now as we have already found out the files we will play with them, lets delete them first. This will be majorly helpful when we want to delete the old logs which are a month old. In this example we are deleting files older than 15 days.

find *.* -mtime -15 -exec rm -rf {}\;

7. Finding a pattern:

Now if we want to find a particular pattern in all files using the find command we can certainly use it to our ease. In the below example we will find the pattern sheron.

find . -type f -exec grep “sheron” {} \; -print

8. Finding and replacing:

Now we can replace the pattern we have found with multiple options, one of them is using perl. In the below example we will replace the pattern sheron with pattern gerard using perl.

find /path/to/start/from/ -type f | xargs perl -pi -e ‘s/sheron/gerard/g’

9. Finding and replacing in multiple files:

The below example finds the pattern sheron in all text files and then in all those files replaces the pattern sheron with the pattern gerard with the use of strean editor. The grep -il statement finds all instances of gerard in files ending in txt and i = ignore case, l = only list the filename. The file names are passed to sed, which runs a regular expression to change all instances of sheron to gerard. Since sed doesn’t overwrite a file, I redirected the output to a temp file and then renamed it back to the original file name.

#!/bin/sh
for file in $(grep -il “Sheron” *.txt)
do
sed -e “s/Sheron/Gerard/ig” $file > /tmp/tempfile.tmp
mv /tmp/tempfile.tmp $file
done.

10. Deleting the carrot M character:

Now our very own find can also be used to t replace ^M character which is introduced in our files when we export files from windows to Unix. Now the below example uses a simple loop to do the same.

for i in `find . -type f` ; do dos2unix $i $i; done

Enjoy playing with files and patterns

The Broadcom Wireless Card Guide

Introduction: Unfortunately, most Broadcom wireless cards need proprietary firmware to get connected; many major distributions make it easy to get this firmware, but the tools they bundle need an Internet connection to get it, which is not always possible. Most tutorials currently online also assume you have a working Internet connection already, but this one will give you instructions on how to install the drivers manually. Don’t worry, it’s not that hard. :D

Step 1: find out what card you have.
There are lots of Broadcom cards out there. To find out what yours is, open a terminal and type or copy-paste (use right-click > Paste to paste into terminals) the following for Ubuntu and Debian:
sudo lspci -vnn | grep 14e4
Or the following for Fedora and others:
su -c "lspci -vnn | grep 14e4
The result should be something like this:
02:00.0 Network controller [0280]: Broadcom Corporation BCM4312 802.11b/g LP-PHY [14e4:4315] (rev 01)
From this, you need the PCI ID, which has been italicized in the sample above. Now look up your PCI ID in the following table:

PCI ID Chip Driver(s) Notes
14e4:4301 BCM4303 b43legacy -
14e4:4306 BCM4306 rev. 2 b43legacy Two IDs, same card (?)
14e4:4320
14e4:4307 BCM4306 rev. 3 b43 -
14e4:4311 BCM4311 b43, wl -
14e4:4312 BCM4312 b43 Unsure about this one
14e4:4313 BCM4313 b43, wl Unsure about this one
14e4:4315 BCM4312 b43, wl Requires LP/PHY firmware
14e4:4318 BCM4318 b43 -
14e4:4328 BCM4321 wl Two IDs, same card (?)
14e4:4329
14e4:432b BCM4322 wl -
14e4:4353 BCM43224 wl -
14e4:4357 BCM43225 wl -

Okay, that table above might confuse you, so let me explain. On the first column you get the PCI ID of the card, which you should already know. On the second column, you get the actual chip’s name. These are not the same as Product IDs (the part of the PCI ID after 14e4:); you can probably use those to verify that you have the right card. On the third column you have the drivers that support each card. You will need to pick one of them and install it; details lie below. And on the fourth column you have any notes related to the card.
If your card isn’t listed there, it probably doesn’t work. I recommend you check this page for more details, but watch out for conflicting information!

Step 2: Pick the right driver.
There are two major drivers for the current series of Broadcom cards, plus one for older cards and one in development (not mentioned here).

  1. Broadcom STA (also known as wl) is the official, Broadcom driver for a few cards typically found in Dell and HP laptops. It’s theoretically more stable than b43 and doesn’t need additional firmware, but it lacks some advanced features and is proprietary software.
  2. b43 is a free-as-in-freedom unofficial driver that requires, however, proprietary firmware to work. It offers a lot more features than wl and, in my experience, is also a lot faster. It can have some issues with new cards, but it should generally work.
  3. b43-legacy is a driver for older BCM43xx cards that don’t work with b43; it’s also free-as-in-freedom, but requires proprietary firmware.

Using this information and the table above, pick one of the drivers and skip to the relevant section…

Installing the Broadcom STA (wl) driver
Ubuntu: you’ll need to download the following packages (choose your Ubuntu release, architecture and mirror): dkms, patch, fakeroot and bcmwl-kernel-source.
After you’ve downloaded the .deb files, use a USB flashdrive or some other means of file transfer to get them on the computer you want to install to. Then, just double-clicking each file should bring up Ubuntu Software Center and let you install it.
Then just navigate to System > Administration > Additional Drivers (or Hardware Drivers for older releases) and activate the Broadcom STA driver.
Check out this page for more information about online installation, but please note that the offline installation instructions are pretty much outdated for wl.
Important note: if this fails, use Keryx (0.92, not 1.0) to do an offline upgrade and install the bcmwl-kernel-source package. An up-to-date Keryx tutorial is on my to-do list.
Other distributions: you can compile the STA driver yourself. 32-bit .tar.gz, 64-bit .tar.gz, readme. The readme explains how to compile the drivers pretty well, so I’m not going to do that here.

Installing the b43 driver (non-LP/PHY cards)
If your card’s PCI ID is 14e4:4315 (BCM4312 chip), you should use the instructions a little further down, since it’s a LP/PHY card.
Get b43-fwcutter: for Ubuntu, download and install b43-fwcutter (pick your version and architecture; direct link for the impatient). I believe it’s installed by default with openSUSE and Fedora, and possibly others. If your distribution doesn’t have it, download the tarball and compile it using the following commands (I recommend you only do this if you have some experience with the Linux shell):
tar xjf b43-fwcutter-013.tar.bz2
cd b43-fwcutter-013
make
cd ..

Unload the module: run this command: sudo modprobe -rv b43 ssb (or su -c "modprobe -rv b43 ssb" if you don’t have sudo installed or usable).
Get the firmware: you’ll need to download this package. Then open a Terminal, cd to where you downloaded it and run the following commands (substitute b43-fwcutter with the actual path to the b43-fwcutter executable if you compiled it yourself):
tar xjf broadcom-wl-4.150.10.5.tar.bz2
cd broadcom-wl-4.150.10.5/driver
sudo b43-fwcutter -w /lib/firmware wl_apsta_mimo.o

If you don’t have sudo installed or usable (just su), use this command instead:
su -c "b43-fwcutter -w /lib/firmware wl_apsta_mimo.o"
Finish up: I suggest you restart. You can probably get around with using sudo modprobe -v b43 on some distros (I usually do that on Ubuntu), but please, please, don’t ever run that command on openSUSE, you’ll just completely mess your system up and yell at me. On second thought, please just restart after installing.

Installing the b43 driver with LP/PHY firmware
This seems to only be relevant for BCM4312, which happens to be the card I have. First, you’ll need b43-fwcutter, which you can get using the instructions above. Then, get the correct firmware package and use these commands to install it:
sudo modprobe -rv b43 ssb
tar xjf broadcom-wl-4.178.10.4.tar.bz2
cd broadcom-wl-4.178.10.4/linux
sudo b43-fwcutter -w /lib/firmware wl_apsta.o

As usual, change sudo to su -c and put the rest in quotes if you don’t have sudo.
To finish up, restart your system.

Installing the b43legacy driver
Again, get b43-fwcutter as described above, then download this package and use the following command to install it:
sudo ./b43-fwcutter-013/b43-fwcutter -w /lib/firmware wl_apsta-3.130.20.0.o
I don’t own any b43legacy-compatible card, so I’m not sure whether you need to unload the b43legacy module (using sudo modprobe -rv b43legacy before you install the firmware. I’d say you should do it, just to be sure, then restart after installing the firmware.

Linux create your own GnuPG private and public key

GNU gpg is encryption and signing tool. The GNU Privacy Guard (GnuPG or GPG) is a free software replacement for the PGP suite of cryptographic software. GnuPG encrypts messages using asymmetric keypairs individually generated by GnuPG users. The resulting public keys can be exchanged with other users in a variety of ways, such as Internet key servers. They must always be exchanged carefully to prevent identity spoofing by corrupting public key ↔ ‘owner’ identity correspondences. It is also possible to add a cryptographic digital signature to a message, so the message integrity and sender can be verified, if a particular correspondence relied upon has not been corrupted.

How do I create my own GnuPG private and public key

  1. Login to your shell account
  2. Use gpg command to create the keys
  3. $ gpg --gen-key

    Output:

    gpg (GnuPG) 1.4.1; Copyright (C) 2005 Free Software Foundation, Inc.
    This program comes with ABSOLUTELY NO WARRANTY.
    This is free software, and you are welcome to redistribute it
    under certain conditions. See the file COPYING for details.
    
    gpg: directory `/home/sheron/.gnupg' created
    gpg: new configuration file `/home/sheron/.gnupg/gpg.conf' created
    gpg: WARNING: options in `/home/sheron/.gnupg/gpg.conf' are not yet active during this run
    gpg: keyring `/home/sheron/.gnupg/secring.gpg' created
    gpg: keyring `/home/sheron/.gnupg/pubring.gpg' created
    Please select what kind of key you want:
       (1) DSA and Elgamal (default)
       (2) DSA (sign only)
       (5) RSA (sign only)
    Your selection? Press [Enter] Key
    DSA keypair will have 1024 bits.
    ELG-E keys may be between 1024 and 4096 bits long.
    What keysize do you want? (2048) Press [Enter] Key
    Requested keysize is 2048 bits
    Please specify how long the key should be valid.
             0 = key does not expire
            = key expires in n days
          w = key expires in n weeks
          m = key expires in n months
          y = key expires in n years
    Key is valid for? (0) Press [Enter] Key
    Key does not expire at all
    Is this correct? (y/N) y
    
    You need a user ID to identify your key; the software constructs the user ID
    from the Real Name, Comment and Email Address in this form:
        "Heinrich Heine (Der Dichter) "
    
    Real name: sheron
    Email address: lgjsheron@gmail.com
    Comment:[Enter] key
    You selected this USER-ID:
        "sheron"
    
    Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
    You need a Passphrase to protect your secret key.
    
    Enter passphrase: [Enter password twice]
    We need to generate a lot of random bytes. It is a good idea to perform
    some other action (type on the keyboard, move the mouse, utilize the
    disks) during the prime generation; this gives the random number
    generator a better chance to gain enough entropy.
    .+++++....+++++++++++++++..++++++++++..++++++++++...++++++++++++++++++++.+++++++++++++++++++++++++++++++++++.+++++..++++++++++++++++++++.++++++++++..+++++.+++++.+++++>+++++....+++++>.+++++...................................................................+++++^^^^^^^^^^^
    gpg: /home/sheron/.gnupg/trustdb.gpg: trustdb created
    gpg: key 8E19F126 marked as ultimately trusted
    public and secret key created and signed.
    
    gpg: checking the trustdb
    gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
    gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
    pub   1024D/8E19F126 2007-02-10
          Key fingerprint = A7AF E25D 3E8D 6946 37CC  8CCE 12C4 8DC1 8E19 F126
    uid                  sheron 
    sub   2048g/032824B9 2007-02-10
  4. Now keys generated, you can list your own key using:
  5. $ gpg -K

    OR

    $ gpg --list-keys

    Output:

    /home/sheron/.gnupg/pubring.gpg
    ------------------------------
    pub   1024D/CA7A8402 2007-02-10
    uid    sheron
    sub   2048g/0A7B4F93 2007-02-10
    

    Let us try to understand the line

    pub 1024D/CA7A8402 2007-02-10:

    pub : Public key
    1024D : The number of bits in the key
    CA7A8402 : The key ID
    2007-02-10 : The date of key creation
    sheron : The user real name
    <lgjsheron@gmail.com> :The email id

    Most important is the key ID i.e. CA7A8402.

    Make sure you use powerful passphrase to protect keys and not the easy one.

  6. To list secret key, type the command:
  7. $ gpg --list-secret-keys

    Output:

    /home/sheron/.gnupg/secring.gpg
    ------------------------------
    sec   1024D/CA7A8402 2007-02-10
    uid                 sheron
    ssb   2048g/0A7B4F93 2007-02-10

How to install firefox 4 in ubuntu using PPA

Go to Applications > Ubuntu Software Center from the top panel go to Edit > Software Sources and click the ‘Other Software’ tab.Press ‘Add’ button and then paste the following line

ppa:mozillateam/firefox-stable

After adding the PPA you will be prompted to update your sources.

Once the update is done you can head to System > Administration > Update Manager to perform an upgrade.

Install firefox 4 using terminal

Go to Applications > Terminal in the terminal just enter the following commands

sudo add-apt-repository ppa:mozillateam/firefox-stable

sudo apt-get update

sudo apt-get upgrade

The Beginner’s Guide to Linux Disk Utilities

Knowing how to check the condition of your hard disk is useful to determine when to replace your hard disk. In today’s article, we will show you some Linux disk utilities to diagnose the health of your hard disk.

Image by Scoobay

S.M.A.R.T System

Most modern ATA and SCSI hard disks have a Self-Monitoring, Analysis, and Reporting Technology (SMART) system. SMART hard disks internally monitor their own health and performance.

The SMART tool assesses the condition of your hard disk based on: the throughput of the hard disk, the seek errors rate of the magnetic heads, and other attributes that your hard disk manufacturer built into their hard disk.

Most implementations of SMART systems allow users to perform self-tests to monitor the performance and reliability of their hard disks. The simplest way to perform a SMART system test with Ubuntu is using the ‘Disk Utility’ under the ‘System’ > ‘Administration’ menu.



The disk utility lets you see the model, serial number, firmware, and the overall health assessment of the hard disk, as well as whether a SMART system is enabled on the hard disk.

The ‘SMART data’ button lets you see the SMART features of your hard disk.

The ‘Run Self-test’ button lets you initiate a short,extended, or a conveyance self-test on the hard disk.

When you execute these tests, you’ll see a progress meter, letting you see how far through the test is and what the estimated time of completion is.

The ‘Attributed section’ lets you see the errors and self-test information.

File System Check

There some other tools, beside the Disk Utility GUI, that we can use to diagnose the health of our hard disk. The File System Check (FSCK), that only comes as a command line tool, is one of the tools that we often use to check the condition of our hard disk.

You can use the ‘Check Filesystem’ feature of the ‘Disk Utility’ to perform the same check,if you are not a command line geek like us.

Of course, there are some situations where we have to use the command line tool to check our file system. For example when we are using a headless system, when our Linux box fails to boot, or when we simply want to show off our command line Kungfu skills to our friends.

At first, the FSCK command line tool looks like something that only a computer geek can handle; But you will find that FSCK is a very easy tool to use. There is one thing to note before you run FSCK; You need to unmount the file system using the ‘umount’ command. Fixing a mounted file system with FSCK could end up creating more damage than the original problem.

sudo umount /dev/sdb

The FSCK command is pretty straightforward:

sudo fsck -t ext4 /dev/sdb 

This command checks an ext4 file system (/dev/sdb) for inconsistencies. You should replace /dev/sdb with your own partition. You can run the ‘fdisk’ command to find out your system partitions:

sudo fdisk -l

Scheduled File System Checks

If you’re using Ubuntu, you will notice that Ubuntu runs an FSCK session when you boot your system from time to time. If you find this scheduled check annoying, you can re-schedule the scan using the ‘tune2fs’ command. Here’s how it typically looks like:

The mount count parameter tells us that Ubuntu scans our hard disk after 33 disk mounts.

We can configure the mount count using the ‘-c’ option:

sudo tune2fs -c 35 /dev/sda1

This command will re-configure Ubuntu to scan our hard disk after 35 hard disk mounts when the system boots.

Note: change ‘/dev/sda1/’ with your own partition

Bad Blocks

A bad sector is a sector on a computer’s disk drive that cannot be used due to permanent damage (or an OS inability to successfully access it), such as physical damage to the disk surface.

There are two ways to detect bad sectors in Linux: you can use the Disk Utility GUI, or if you are a command line geek like us, you can use the badblocks command to check your hard disk for bad sectors:

sudo badblocks -v /dev/sdb1

Badblock will give us the number of bad sectors in our hard disk.

zainul@zainul-laptop:~$ sudo badblocks -v /dev/sdb1
Checking blocks 0 to 97683200
Checking for bad blocks (read-only test): 3134528 done, 3:27 elapsed
3134560 done, 8:33 elapsed
3134561 done, 10:15 elapsed
3134562 done, 11:57 elapsed
3134563 done, 13:39 elapsed
done
Pass completed, 5 bad blocks found.

You have two options when you see bad blocks. You can either look for a new hard disk, or mark these bad blocks as unusable hard disk sectors. This involves two steps:

First we have to write the location of the bad sectors into a flat file.

sudo badblocks /dev/sdb > /home/zainul/bad-blocks

After that, we need to feed the flat file into the FSCK command to mark these bad sectors as ‘unusable’ sectors.

sudo fsck -l bad-blocks /dev/sdb

FSCK, Badblocks, and Disk Utility are some of the disk utilities that we often use to scan our hard disks. Do share with the other fellow readers if you know other Linux disk utilities to scan hard disks.

LibreCAD: open source 2D CAD software

LibreCAD is a free open source 2D CAD drawing software, formerly known as CADuntu. It’s based on the community edition of QCad ported to Qt4 and works natively on OSX, Windows and Linux. And it can easily be installed in Ubuntu, getting a PPA for Lucid, Maverick and Natty.
LibreCAD: open source 2D CAD software
Install LibreCAD in Ubuntu:

For Ubuntu 10.04:

sudo add-apt-repository ppa:showard314/librecad
sudo apt-get update
sudo apt-get install librecad

For Ubuntu 10.10 and 11.04:

sudo add-apt-repository ppa:showard314/ppa
sudo apt-get update
sudo apt-get install librecad