Weblogic server High CPU Utilization

An error (weblogic.common.resourcepool.ResourceDisabledException: Pool BAMDataSource is

Suspended, cannot allocate resources to applications..) has occurred in the Active Data Cache
storage system
Steps:

1. top command > to know the high CPU consuming process.

2. ps -ef|grep > to know details of the process consuming
high CPU.
3. ls -lrt /proc//fd
or ls -lrt /proc//fd |grep log -> to know the files
accessed by the process. To know the log files and other
related files.
4. Check the log files, out files etc to get the related
error.
what is the command for,
1. find logs of last 5 days
2. find logs of last 5th day
3. find logs of last 10 to 20 days.
4. find logs of particular day(15aug2014).
5. find logs of particular period (15th aug to 20th aug)
Pack and Unpack Commands
/opt/bea/products/middleware/wlserver/common/bin/pack.sh -domain="/wls_domains/wlsapp01" template="/wls_domains/wlsapp01/template/Admin.jar" -template_name="WLS CLUSTER DOMAIN" log=/wls_domains/wlsapp01/logs/wlsPack.log -log_priority=INFO

/opt/bea/products/middleware/wlserver/common/bin/unpack.sh -template=/wls_domains/Admin.jar
-user_name=weblogic -password=P@55world -domain=/wls_domains/wlsapp01 -log=unpack.log log_priority=INFO

1. find logs of last 5 days find . -name xxxxx.log -mtime -5

2. find logs of last 5th day find . -name xxxxx.log -mtime 5
3. find logs of last 10 to 20 days find . -name xxxxx.log -mtime +10 -mtime -20
4. find logs of particular day(15aug2014). find . -type f -mtime $(( ( $(date +%s) - $(date -d '2014-0815' +%s) ) / 60 / 60 / 24 - 1 ))
5. find logs of particular period (15th aug to 20th aug) find . -type f -mtime +$(( ( $(date +%s) - $(date
-d '2014-08-15' +%s) ) / 60 / 60 / 24 - 1 )) -mtime -$(( ( $(date +%s) - $(date -d '2014-08-15' +%s) ) /
60 / 60 / 24 - 1 ))
grep -lr -e 'DEBUG' * | grep -v log | grep -v txt| grep -v nohup | xargs sed -i 's/DEBUG/INFO/g'
grep -lr -e '10.1' * | grep -v log | grep -v txt| grep -v nohup | xargs sed -i 's/10.1/10.2/g'

grep -lr -e '10.1.1' * | grep -v log | grep -v txt| grep -v nohup | xargs sed -i 's/10.1.1/10.2.1/g'
grep -lr -e 'btvhydct1l' * | grep -v log | grep -v txt| grep -v nohup | xargs sed -i
's/btvhydct1l/btvhydct2l/g'
grep -lr -e '2403645797' * | grep -v log | grep -v txt| grep -v nohup | xargs sed -i
's/2403645797/2419782787/g'
MY UAN : UAN - 100361712567
GN/GGN/5572/000/216589

2688692
http://www.tecmint.com/use-pam_tally2-to-lock-and-unlock-ssh-failed-login-attempts/
VOL D root password : Carn@ti0n
### command to format an xml file
xmllint --format
## to convert file to dos2unix if package not installed
sed -i 's/\r//' filename
### Command to know whether a machine is virtual or physical
dmidecode | grep -i "Virtual"
eg o/p:
Version: VirtualBox
Product Name: VirtualBox
Family: Virtual Machine
--- Command to replace a string or path with other one--:%s/\/wls_domains\/Datamart\//\/Datamart\//gc
=== Command to know the DB version in linux ===
login to the DB from sqlplus then give below
29Ph05il69BM
select * from v$version;
---------------------------

alter user dmt_user identified by dmt_user; ==> to reset the password in DB

----------------------grep -lr -e '2403645797' * | grep -v log | grep -v txt| grep -v nohup
grep -lr -e '2403646379' * | grep -v log | grep -v txt| grep -v nohup
vgextend rootvg /dev/sdc1 ==> it will extend the disk space of rootvg from /dev/sdc1
ps -ef | grep username | awk '{ print $2 }' == Command to get the process id's of single user
ps -ef |grep wlsms01 | awk '{print $2}' | xargs kill -9 == same command to kill all pid's of specific user
service rpcbind status|stop|start|restart
service nfs restart|start|stop|status
service portmap restart|start|stop|status
/etc/network
=== Command to resolve the permission denied issue with mounts and to hit on source server
please check if any files are there in /proc/fs/nfsd
mount -t nfsd none /proc/fs/nfsd
rm101prx01.room101.com/8080
============================================
Heap DUMP Command :
/opt/bea/jdk1.7.0_03/bin/jmap -dump:format=b,file=filename.bin PID
## java install command
==========================================================
need to create a soft link in the below path -cd /usr/bin/
ln -s /opt/bea/jdk1.7.0_45/bin/java .
export JAVA_HOME=/opt/bea/jdk1.7.0_45/bin/java
export PATH=/usr/bin:$JAVA_HOME
which version
cd /usr/bin/
ln -s /opt/bea/jdk1.7.0_45/bin/java .
ls -ltr
cd /usr/bin/
chown -R bea.beagrp java
ls -ltr jav*
==============================================================
find . -type f|xargs perl -p -i -e 's, string1,string2'
grep -lr -e 'searchString' * | grep -v log | grep -v txt| grep -v nohup | xargs sed -i
's/searchString/replaceString/g'

==============================================================
-------------------------------------------------------------How to know which ports are already used for environment??
"netstat -nap"
------------------------------------------------------------> to unlock unix user
pam_tally --reset --user username
pam_tally2 --user wlsapp01 --reset
=============================
chage -l username ==> will give the useraccount details.
chage -I -1 -m 0 -M 99999 -E -1 user ==> this will set the user's password/account to expire to never
chsh -s /bin/bash username ===> command to give login permission to a user
faillog -u username -> to view the failog for that user
faillog -r username -> to remove the fail logins in a server.
***************************
-I -1 : This will set the Password inactive to never
-m 0 : This will set the minimum number of days between password change to 0
-M 99999 : This will set the maximum number of days between password change to 99999
-E -1 : This will set Account expires to never.
This will disable the password expiry of a user if it is already enabled.
*************************************************
command to give nologin to existing user
/usr/sbin/usermod -s /sbin/nologin username
command to delete a user
/usr/sbin/usermod -r username {-r => remove user home dir}
Command to set home directory in linux..
(login as root)
usermod -d /new/path/ username
==============================================
#####
TO add a existing user to a group
groupadd --gid 131 grpname
#####
********
to login as root user we need to edit the /etc/ssh/sshd_config -> change PermitRootLogin to 'yes'
and save.
Then give 'service sshd restart'

LM VIP-sys0= 10.101.43.250
http://www.thegeekstuff.com/2012/08/fsck-command-examples/
ifconfig eth0 mtu 1500 up
/etc/sysconfig/network-scripts/ifcfg-eth0
fsck -AR -y
======================================================
##LogsCleanUp
find . -type f -mtime +3 -exec rm -f {} \;
find . -type f -mtime +30 -exec rm -rf {} \;
00 15 * * * find /wls_domains/wlsmda01_logs -type f -mtime +3 -name "*.*" -exec rm -f {} \;
00 15 * * * find /wls_domains/wlsmda01/servers/managed1_wlsmda01/logs -type f -mtime +3 name "*.*" -exec rm -f {} \;
find /wls_domains/solrupi/SOLR_UPI/solr/ControlFile -name "userProfile_*_Ctrl.txt" -mtime +30 exec rm -rf {} \;
find /reports/filter/PackagingInput -name "ContentFeed_*" -mtime +5 -exec rm -rf {} \;
find /reports/filter/PackagingInput/lowLatency -name "ContentFeed_*" -mtime +5 -exec rm -rf {} \;
==== logs deleting script ====
find /wls_domains/wlslss01/LSS/config/csv -name "TVE_Linear_Blackouts_*.csv" -mtime +2 -exec
mv {} /wls_domains/wlslss01/LSS/config/archive \;
======================================================
.... Apache Checks ....
service httpd status
service httpd restart
netstat -an|grep 443
### To create a physical bind between a umounted folder to a mounted file system ####
mount -o bind /old_directory /new_directoty
=======Weblogic Managed Server JVM StartUp parameters sample======
MEM_ARGS="-Xms2150m -Xmx2150m -XX:NewSize=256m -XX:MaxNewSize=716m XX:PermSize=256m -XX:MaxPermSize=512m -XX:SurvivorRatio=6 -XX:TargetSurvivorRatio=90 XX:MaxTenuringThreshold=15 -XX:+UseBiasedLocking -XX:+UseParNewGC XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -verbose:gc -XX:+PrintGCDetails XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution Xloggc:/wls_domains/wlsapp02_logs/LDS_GC.log"
====Password less connectivy b/w two servers/users======
1. ssh-keygen -t rsa [source server]
2. ssh-copy-id -i ~/.ssh/id_rsa.pub dptapp@10.101.41.38 on [source server]
3. dptapp@10.101.41.38 on [source server]

In Server 1
1. ssh-keygen -t rsa
a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa):
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A
2. Now use ssh to create a directory ~/.ssh as user b on B. (The directory may already exist, which is
fine):
a@A:~> ssh b'susername@B's-IpAddress mkdir -p .ssh
b@B's password:
3. both servers Home directories should be with 700 permissions
4. a@A:~> cat .ssh/id_rsa.pub | ssh b'susername@B's-IpAddress 'cat >> .ssh/authorized_keys'
b@B's password:
5. From now on you can log into B as b from A without password:
a@A:~> ssh b'susername@B's-IpAddress
Example :chmod 700 /u01/oracle11
ssh-keygen -t rsa
ssh dptapp@10.101.41.38 mkdir -p .ssh
cat .ssh/id_rsa.pub | ssh dptapp@10.101.41.38 'cat >> .ssh/authorized_keys'
ssh dptapp@10.101.41.38
===============================================
before executing below command, run wlssetEnv.sh file in WL_HOME/server/bin/ folder
java weblogic.Admin -url http://10.101.20.168:50300 -username weblogic -password P@55world
UNLOCK
if above command doesn't works...
set the Env first and run again
sh /opt/bea/wlserver_10.3/server/bin/setWLSEnv.sh
java -jar /usr/lib/jenkins/jenkins.war --httpPort=50100 &
===============================

How to check on which port apache is running????

Answers :"lsof -i" list open ports and the corresponding applications.
"ps aux | grep apache2" For a general check if an app is running you could just use
"ps aux | grep httpd" for redhat/centos
"netstat -anp | grep apache" or "sudo netstat -anp | grep apache"
"lsof | grep httpd" or "lsof | grep apache"
"netstat -tulpn" or "sockstat | grep apache httpd" for FreeBSD
====================================================
Please follow the steps in the below url to configure/setup ftp service in linux machines
http://computernetworkingnotes.com/network-administration/how-to-configure-ftp-server-inrhel6.html
====================================================
http://stackoverflow.com/questions/12076326/how-to-install-maven2-on-redhat-linux
====================================================
Below is the command for generating the trust key :
keytool -import -trustcacerts -alias mykey -keystore server_trust.jks -file server.crt
keytool -import -trustcacerts -file /path/to/ca/ca.pem -alias CA_ALIAS -keystore
$JAVA_HOME/jre/lib/security/cacerts
=================================================
Startup command for DTP script
nohup java -DDTPProp=/wls_domains/dtpuser01/MnB/DTP/JAR/DirectToPlatform.properties classpath .:log4j-1.2.9.jar:commons-lang-2.4.jar:DirectToPlatform.jar directtoplatform.Main &
Puppet commands
========->
mco facts lsbdistrelease -F ipaddress=10.101.43.84 -F ipaddress=10.101.43.85 -F
ipaddress=10.101.43.86

echo $HOME
echo $MAVEN_OPTS
echo $M2_HOME
echo $JAVA_HOME
echo $M2

export M2_HOME=/cruisecontrol/maven/apache-maven-3.0.5
export MAVEN_OPTS="-Xms256m -Xmx512m"
export JAVA_HOME=/opt/bea/jdk1.7.0_45
export HOME=/cruisecontrol
export M2=/cruisecontrol/maven/apache-maven-3.0.5/bin

----SAR Command---SAR - System Activity Report

------------------------LM ST_1 crocus curl URL===>
curl -v -k -H "Authorization: Basic YnRjcm9jdXM6Y3JvY3VzdXNlcjEyMw=="
"https://10.101.41.224:50200/crocus-stub/subscriber/token?IP=25.10.25.10"
====
command to import .crt file to cacerts in java
keytool -import -trustcacerts -alias ca_crocusStub -file server.crt -keystore cacerts
Password : changeit
=====
java DConfig=/wls_domains/dptapp/EntitlementNormalizationScript/config/NormalisationScriptProperti
es.properties DLogConfigPath=/wls_domains/dptapp/EntitlementNormalizationScript/config/NormalisationLog4jP
roperties.properties -jar
/wls_domains/dptapp/EntitlementNormalizationScript/jar/EntitlementNormalisation.jar
/wls_domains/dptapp/EntitlementNormalizationScript/inputFile.csv 100 &
=====
Please use the above script to run the time stamp update which takes jsonfile as input
-bash-3.2$ python rtman_json_update.py
/opt/tomcat/Stubs/config/test/RTMAN_RetrieveIpAddressData.JSON

echo "- - -" > /sys/class/scsi_host/host0/scan

fdisk -l
tail -f /var/log/message
### search Indexer command

/usr/bin/perl /opt/btvsearch/indexer/BTVSearchIndexer.pl
/opt/btvsearch/indexer/indexer.properties /portal/btvo/xml/product true
>>/opt/btvsearch/indexer/logs/BTVIndexer.log_$(date +\%y\%m\%d) 2>&1

### XFB start commands ###

export CFTBTCSCRIPD=/osmf/mgmt/condir/cft/BT/central/script;
cd /osmf/mgmt/condir/cft/cft/Transfer_CFT/home/bin
./cftstart
telnet 10.101.41.38 1761
=======
SYS-0 COuchbase password:
C0uc#bas3
LetMeIn1! - ST-1,CIT-1
Cluster1 - rm101sys0lapp80/81/82/83 pwd = Cbsys01
Cluster2 - rm101sys0lapp84/85/91/92 pwd = Cbsys02
SYS-1 COuchbase password:
Cbsys1
###
service couchbase-server restart
=======
###Command to start jetty service in media server
service watchfolder-agent-service start

======puppet agent installation steps====

curl -k https://10.101.20.160:8140/packages/current/install.bash | bash
====== OS patch steps ====
export http_proxy=http://rm101prx01.room101.com:8080
export https_proxy=http://rm101prx01.room101.com:8080
wget http://kickstart-yum.dci.bt.com/patching/Patch_OEL.sh
chmod +x Patch_OEL.sh
./Patch_OEL.sh 03-2016 PATCH
#####GIT code download command
git clone -b dev git@10.101.20.137:puppet/mplay.git

kazamaGIRI7$
if [ -r "$DOMAIN_HOME/bin/bt_applications_inc.sh" ]; then
. $DOMAIN_HOME/bin/bt_applications_inc.sh
Fi

http://metasolv.blogspot.in/2015_12_01_archive.html
http://middlewaresnippets.blogspot.in/2014/06/installing-and-configuring-java.html
https://coderanch.com/forums/f-40/BEA-Weblogic
http://learn-weblogic.blogspot.in/2013/07/weblogic-virtual-host-working-with.html
http://weblogic-tips.com/2012/01/24/weblogic-deployment-staging-modes/
https://techworldrocks.wordpress.com/2014/04/04/differences-between-wls-11g-and-12c-12-1-2/
http://www.allinterview.com/showanswers/32111/what-is-virtual-host-in-weblogic-how-to-createit-what-is-the-advantage.htm
http://narayanasetti.blogspot.in/2014/03/what-is-outofmemory-in-permgen-space.html
http://websphereinterviewquestions.blogspot.in/2009/12/diagnosing-out-of-memory-errorsand.html
http://helpdesk.arcturustech.com/entries/28626098-Weblogic-Coherence-Cluster-Setup-Applicare
http://www.sadgurutechnologies.com/course-detail.php?id=3