Quantcast
Channel: Karl Arao's TiddlyWiki
Viewing all 1150 articles
Browse latest View live

SolarisNetwork Issues

$
0
0


corrupted MAC

ssh or scp connection terminates with the error "Corrupted MAC on input" (Doc ID 1389880.1) To BottomTo Bottom

NFS mount hang

System gets hung while reboot due to in progress NFS READ or WRITE operations, even though NFS server is available https://access.redhat.com/solutions/778173
RHEL mount hangs: nfs: server [...] not responding, still trying https://access.redhat.com/solutions/28211
Strange NFS problem (not responding still trying) https://community.hpe.com/t5/System-Administration/Strange-NFS-problem-not-responding-still-trying/td-p/3269194





SolarisNetwork configure network

SolarisNetwork

ACID, CAP theorem, BASE, NRW notation, BigData 4Vs + 1

$
0
0
Further Reading:Brewer (http://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed) and Gilbert and Lynch (http://groups.csail.mit.edu/tds/papers/Gilbert/Brewer2.pdf) on the CAP Theorem; Vogels (http://queue.acm.org/detail.cfm?id=1466448) on Eventual Consistency, Hamilton (http://perspectives.mvdirona.com/2010/02/24/ILoveEventualConsistencyBut.aspx) on its limitations, and Bailis and Ghodsi (https://queue.acm.org/detail.cfm?id=2462076) on measuring it and more; and Sirer (http://hackingdistributed.com/2013/03/23/consistency-alphabet-soup/) on the multiple meanings of consistency in Computer Science. Liveness manifestos (http://cs.nyu.edu/acsys/beyond-safety/liveness.htm) has interesting definition variants for liveness and safety.

Big Data 4Vs + 1

Volume - scale at which data is generated
Variety - different forms of data
Velocity - data arrives in continuous stream
Veracity - uncertainty: data is not always accurate
Value - immediacy and hidden relationships


ACID

  • redo and undo in Oracle provides ACID
Atomic - they all complete successfully, or not at all
Consistent - integrity rules. ACID consistency is all about database rules.
Isolated - locking
Durable - transaction is guaranteed

http://cacm.acm.org/magazines/2011/6/108651-10-rules-for-scalable-performance-in-simple-operation-datastores/fulltext
http://www.slideshare.net/jkanagaraj/oracle-vs-nosql-the-good-the-bad-and-the-ugly
http://highscalability.com/blog/2009/11/30/why-existing-databases-rac-are-so-breakable.html
Databases in the wild file:///C:/Users/karl/Downloads/Databases%20in%20the%20Wild%20(1).pdf




CAP theorem

  • CAP is a tool to explain trade-offs in distributed systems.
Consistent: All replicas of the same data will be the same value across a distributed system. CAP consistency promises that every replica of the same logical value, spread across nodes in a distributed system, has the same exact value at all times. Note that this is a logical guarantee, rather than a physical one. Due to the speed of light, it may take some non-zero time to replicate values across a cluster. The cluster can still present a logical view by preventing clients from viewing different values at different nodes.
Available: All live nodes in a distributed system can process operations and respond to queries.
Partition Tolerant: The system is designed to operate in the face of unplanned network connectivity loss between replicas.
https://en.wikipedia.org/wiki/CAP_theorem
https://dzone.com/articles/better-explaining-cap-theorem
https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html
http://guyharrison.squarespace.com/blog/2010/6/13/consistency-models-in-non-relational-databases.html<- good stuff
http://www.datastax.com/2014/08/comparing-oracle-rac-and-nosql<- good stuff
http://docs.oracle.com/database/121/GSMUG/toc.htm , http://www.oracle.com/technetwork/database/availability/global-data-services-12c-wp-1964780.pdf<- Database Global Data Services Concepts and Administration Guide Global Data Services
http://www.oracle.com/technetwork/database/options/clustering/overview/backtothefuture-2192291.pdf<- good stuff Back to the Future with Oracle Database 12c
https://blogs.oracle.com/MAA/tags/cap<- two parts good stuff
https://www.percona.com/live/mysql-conference-2013/sites/default/files/slides/aslett%20cap%20theorem.pdf<- very good stuff

http://blog.nahurst.com/visual-guide-to-nosql-systems


http://www.ctodigest.com/2014/distributed-applications/the-distributed-relational-database-shattering-the-cap-theorem/
https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed
Spanner, TrueTime and the CAP Theorem https://research.google.com/pubs/pub45855.html , https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45855.pdf
https://www.voltdb.com/blog/disambiguating-acid-and-cap<- difference between two Cs (voltdb founder)
https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html<- nice, a lot of references! author of "Designing Data-Intensive Applications"
https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-reads
http://blog.thislongrun.com/2015/04/cap-availability-high-availability-and_16.html
https://github.com/jepsen-io/knossos
https://aphyr.com/posts/288-the-network-is-reliable
http://dbmsmusings.blogspot.co.uk/2010/04/problems-with-cap-and-yahoos-little.html
https://codahale.com/you-cant-sacrifice-partition-tolerance/
https://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/
http://henryr.github.io/cap-faq/
http://henryr.github.io/distributed-systems-readings/
http://blog.thislongrun.com/2015/03/the-confusing-cap-and-acid-wording.html
https://news.ycombinator.com/item?id=9285751

http://www.slideshare.net/AerospikeDB/acid-cap-aerospike
Next Generation Databases: NoSQL, NewSQL, and Big Data https://www.safaribooksonline.com/library/view/next-generation-databases/9781484213292/9781484213308_Ch09.xhtml#Sec2
https://www.pluralsight.com/courses/cqrs-theory-practice
https://www.pluralsight.com/blog/software-development/relational-non-relational-databases
https://www.amazon.com/Seven-Concurrency-Models-Weeks-Programmers-ebook/dp/B00MH6EMN6/ref=mt_kindle?_encoding=UTF8&me=
https://en.wikipedia.org/wiki/Michael_Stonebraker#Data_Analysis_.26_Extraction
http://scaledb.blogspot.com/2011/03/cap-theorem-event-horizon.html



BASE (eventual consistency)

BASE – Basically Available Soft-state Eventually consistent is an acronym used to contrast this approach with the RDBMS ACID transactions described above.
http://www.allthingsdistributed.com/2008/12/eventually_consistent.html<- amazon cto



NRW notation

NRW notation describes at a high level how a distributed database will trade off consistency, read performance and write performance. NRW stands for:
N: the number of copies of each data item that the database will maintain.
R: the number of copies that the application will access when reading the data item
W: the number of copies of the data item that must be written before the write can complete.


Solaris change zone hostname

$
0
0
zoneadm list -civ | grep er2zgrc319v
zoneadm list -civ | grep er2zgrc320v
zoneadm list -civ | grep er2zgrc321v
zoneadm list -civ | grep er2zgrc322v




# 1 - To check the current environment properties:

    svccfg -s system/identity:node listprop config

    root@er2zgrc321v:~# svccfg -s system/identity:node listprop config
    config                       application
    config/enable_mapping       boolean     true
    config/ignore_dhcp_hostname boolean     false
    config/loopback             astring
    config/nodename             astring     er2zgrc321v


# 2 - Set the new hostname

    from: er2zgrc321v-i
    to: er2zgrc421v

    svccfg -s system/identity:node setprop config/nodename="er2zgrc421v"
    svccfg -s system/identity:node setprop config/loopback="er2zgrc421v"

    root@er2zgrc321v:~# svccfg -s system/identity:node listprop config
    config                       application
    config/enable_mapping       boolean     true
    config/ignore_dhcp_hostname boolean     false
    config/nodename             astring     er2zgrc421v
    config/loopback             astring     er2zgrc421v
    root@er2zgrc321v:~#



# 3- Refresh the properties:

svccfg -s system/identity:node refresh

#4 - Restart the service:

svcadm restart system/identity:node

#5 - verify that the changes took place:

svccfg -s system/identity:node listprop config





zoneadm -z er2zgrc321v-i reboot



root@er2s1app01:~# zoneadm list -civ | grep er2zgrc319v
root@er2s1app01:~# zoneadm list -civ | grep er2zgrc320v
root@er2s1app01:~# zoneadm list -civ | grep er2zgrc321v
  35 er2zgrc321v-i    running     /zones/er2zgrc321v           solaris    excl
root@er2s1app01:~# zoneadm list -civ | grep er2zgrc322v


root@er2s1app01:~# zlogin er2zgrc321v-i
[Connected to zone 'er2zgrc321v-i' pts/5]
Last login: Thu Feb 23 22:17:15 2017 from er2s1vm02.erp.h
Oracle Corporation      SunOS 5.11      11.3    August 2016
You have new mail.
root@er2zgrc421v:~#
root@er2zgrc421v:~#
root@er2zgrc421v:~#
root@er2zgrc421v:~#



SolarisInstallation

Setup X11 Access to the Solaris GUI Gnome Desktop

DistributedComputing


howto create zones

$
0
0
https://docs.oracle.com/cd/E19044-01/sol.containers/817-1592/z.inst.task-1/index.html
https://blogs.oracle.com/mandalika/entry/solaris_10_zone_creation_for

A Technical Overview of Oracle SuperClusterhttp://www.oracle.com/technetwork/server-storage/sun-sparc-enterprise/documentation/o13-045-sc-t5-8-arch-1982476.pdf
Oracle SuperCluster - An Example of Automating the Creating of Application Zones (Doc ID 2041460.1)
Implementing Application Zones on Oracle SuperClusterhttp://www.oracle.com/us/products/servers-storage/servers/sparc/supercluster/wp-app-zones-supercluster-2611960.pdf
Best Practices for Deploying Oracle Solaris Zones with Oracle Database 11g on SPARC SuperClusterhttp://www.oracle.com/technetwork/server-storage/engineered-systems/sparc-supercluster/deploying-zones-11gr2-supercluster-1875864.pdf

https://technicalsanctuary.wordpress.com/2016/04/21/creating-an-application-zone-on-a-supercluster/
https://technicalsanctuary.wordpress.com/2015/05/15/changing-the-number-of-cpus-in-a-zones-processor-set-pooladm-solaris-11-1/
https://technicalsanctuary.wordpress.com/2015/01/07/escaping-out-of-a-zlogin-c/
https://technicalsanctuary.wordpress.com/2014/05/02/solaris-11-repo-which-solaris-releases-are-available/
https://technicalsanctuary.wordpress.com/2014/03/24/showing-the-supercluster-software-release-version/
https://technicalsanctuary.wordpress.com/2011/11/25/check-current-value-of-shmmax-on-solaris/



create a zone in solaris 11

http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-092-s11-zones-intro-524494.html
http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-118-s11-script-zones-524499.html
http://thegeekdiary.com/how-to-create-a-zone-in-solaris-11/
https://richardgood.wordpress.com/2011/11/14/how-to-create-a-zone-in-solaris-11/

mos AIX

$
0
0
Logical Partitioning (LPAR) in AIX
  	Doc ID: 	Note:458571.1


-- INSTALLATION

Minimum Software Versions and Patches Required to Support Oracle Products on IBM pSeries
 	Doc ID:	Note:282036.1

Additional Steps Required To Upgrade IBM JDK On IBM iSeries
 	Doc ID:	Note:457287.1

Questions regarding Oracle database upgrades on AIX
 	Doc ID:	Note:223521.1

PAR: MATRIX IBM AIX for Oracle RDBMS Compatibility
 	Doc ID:	Note:41984.1

http://www-933.ibm.com/eserver/support/fixes/fixcentral/pfixpacks/53



-- BUG on 10.2.0.2 and ML 05 and higher

http://unix.derkeiler.com/Mailing-Lists/AIX-L/2006-09/msg00030.html
http://www.tek-tips.com/viewthread.cfm?qid=1272346&page=7

-- IY fix
ftp://ftp.software.ibm.com/aix/efixes/iy89080/

-- one-of-patch download
https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=5496862&release=80102020&plat_lang=212P&patch_num_id=747092&email=karao@sqlwizard.com&userid=ml-591048.992&

Bug 5496862 - AIX: Mandatory patch to use Oracle with IBM Technology Level 5 (5300-5)
 	Doc ID:	Note:5496862.8

Introduction to "Bug Description" Articles
 	Doc ID:	Note:245840.1

10.2.0.3 PMON CRASHES ON STARTUP ON AIX 5L 5.3 ML05 -- WORKS on ML06
 	Doc ID:	Note:458442.1

Does A DB Running Oracle 10.2.0.2 On Aix 5.3 Tl5 Sp1 Have To Use Patch 5496862
 	Doc ID:	Note:432998.1

Is Patch 5496862 Mandatory for 10.2.0.3?
 	Doc ID:	Note:418105.1

How To Determine Whether an APAR has been Fixed in AIX Version/Maintenance Level
 	Doc ID:	Note:417451.1

How To Determine Service Pack in AIX
 	Doc ID:	Note:421513.1

How Do I Determine The AIX Technology level ?
 	Doc ID:	Note:443343.1

Is Patch 5496862 applicable on AIX 5.3 TL 06 / TL 07 / TL 08 ?
 	Doc ID:	Note:443944.1

How To Determine Whether an APAR has been Fixed in AIX Version/Maintenance Level
 	Doc ID:	Note:417451.1

Is Patch 5496862 Mandatory for 10.2.0.3?
 	Doc ID:	Note:418105.1

How Do I Determine The AIX Technology level ?
 	Doc ID:	Note:443343.1

ORA-01115 ORA-01110 ORA-27091 ORA-27072 Error: 5: I/O error
 	Doc ID:	Note:559697.1

IO Interoperability Issue between IBM ML05 and Oracle Databases
 	Doc ID:	Note:390656.1

Patch 5496862 Now Available For RDBMS Server Version 10.1.0.4.2 (AIX)
 	Doc ID:	Note:418048.1



-- DYNAMIC CPU ALLOCATION BUG

http://oracledoug.com/serendipity/index.php?/archives/1318-10.2.0.2-bug-with-Dynamic-Reconfiguration-of-CPU.html

Bug 4704890 - OERI[kslgetl:1] after adding CPU using dynamic reconfiguration
  	Doc ID: 	Note:4704890.8

Pmon Terminated With Ora-00600 [1100] After Dynamically Increase CPU_COUNT
  	Doc ID: 	Note:467695.1

ORA-600 [kslgetl:1]
  	Doc ID: 	Note:351779.1

Ora-07445 (Internal Error)
  	Doc ID: 	Note:421045.1

Ora-600 [Kslgetl:1] when dynamically changing CPU
  	Doc ID: 	Note:369400.1




-- MEMORY

Memory Consumption on AIX
  	Doc ID: 	259983.1

Solaris

AIX

MainMenu

SolarisAdmin

$
0
0



Installing Oracle Solaris 11

user - jack:jack
root - root:solaris

Two ways of installing

  • using Live CD (only x86) or Text installer

install log location

oracle@enksc1db0201:/export/home/oracle:dbm012
$ less /var/sadm/system/logs/install_log

system messages

less /var/sadm/system/logs/messages 

OBP (openboot prom)

openboot prom <- solaris boot loader used on SPARC
grub <- boot loader used on x86
  • to access the openboot prom
eeprom
monitor
banner


Updating and Managing Packages (IPS)

  • IPS replaced SVR4 found in earlier releases
  • allows you to list, search, install, update, remove packages

IPS admin

    • manage all software packages
    • manage software publishers
    • manage repositories
    • update an image to a new OS release
      • can be used to create images, it's basically an installation and you can basically baseline your image change things with IPS snapshot it or back it up and put it up as a bootable environment and you don't have to make this changes stick you can revert back to the image you had if you like so you can test new OS packages without damaging your system
    • create and manage boot environments
      • this is what the images are - boot environments, you can have a baseline image, or several of them and boot from any of them

IPS terms

    • manifest - describes an IPS package
    • repository - internet or network location, the location is specified by a URI (universal resource identifier)
    • image - a location where IPS packages can be installed
    • catalog - lists all packages in a given repository
    • package archive - file that contains publisher info
    • mirror - repo that contain package content
    • boot environment (BE) - bootable instance of an image (OS)

CLI - IPS


# publisher stuff
pkg publisher    <- list publisher
pkg set-publisher -g http://pkg.openindiana.org/sfe sfe     <- add publisher

# troubleshooting 
pkg info
pkg contents
pkg history
pkg uninstall

beadm (manage boot environments)

$ beadm list
BE               Flags Mountpoint Space   Policy Created
--               ----- ---------- -----   ------ -------
SCMU_2016.07     NR    /          13.65G  static 2016-08-30 15:29
solaris          -     -          115.86M static 2016-08-29 16:55
solaris-backup-1 -     -          103.84M static 2016-08-29 23:02
solaris-bkup     -     -          103.74M static 2016-08-29 22:42
solaris-idr      -     -          299.37M static 2016-08-30 03:57

beadm create test
beadm destroy test 



Administering Services (SMF - service management facility) - svcs and svcadm

  • SMF part of FMA (fault management architecture)
  • comes in both GUI (SMF services) and CLI versions

SMF standard naming convention - FMRI (fault management resource identifier)

  • scheme - type of service
  • location - system service is running on
  • function category - the service function
    • applications
    • network
    • device
    • milestone (run level)
    • system
  • description - service name
  • instance - which instance (if many instances are running)

example name convention

scheme://location/function/description:instance
Example: svc://localhost/network/nfs/server:default

service states

  • online
  • offline
  • disabled
  • maintenance
  • degraded
  • legacy_run

CLI - svcs, svcadm, svccfg

  • svcs - list the services and properties
  • svcadm - administration of services
  • svccfg - create/define your own service (created in a service manifest file, the best is to build from an existing manifest file)

svcs

svcs -a 

# show all services that ExaWatcher depends upon in order to run
$ svcs -d ExaWatcher
STATE          STIME    FMRI
online         Jan_23   svc:/milestone/network:default
online         Jan_23   svc:/system/filesystem/local:default
online         Jan_23   svc:/milestone/multi-user:default
oracle@enksc1db0201:/export/home/oracle:dbm012

# show all services that depend on ExaWatcher itself for them to run
$ svcs -D ExaWatcher
STATE          STIME    FMRI
oracle@enksc1db0201:/export/home/oracle:dbm012


$ svcs -d smtp
STATE          STIME    FMRI
online         Jan_23   svc:/system/identity:domain
online         Jan_23   svc:/network/service:default
online         Jan_23   svc:/milestone/name-services:default
online         Jan_23   svc:/system/filesystem/local:default
online         Jan_23   svc:/system/filesystem/autofs:default
online         Jan_23   svc:/system/system-log:default
oracle@enksc1db0201:/var/svc/log:dbm012
$
oracle@enksc1db0201:/var/svc/log:dbm012
$ svcs -D smtp
STATE          STIME    FMRI
oracle@enksc1db0201:/var/svc/log:dbm012
$
oracle@enksc1db0201:/var/svc/log:dbm012

# verbose on service
$ svcs -xv smtp
svc:/network/smtp:sendmail (sendmail SMTP mail transfer agent)
 State: online since Mon Jan 23 17:12:29 2017
   See: man -M /usr/share/man -s 1M sendmail
   See: /var/svc/log/network-smtp:sendmail.log
Impact: None.



svcadm

svcadm -h

# boot and shutdown system
svcadm milestone svc:/milestone/single-user:default
svcadm milestone svc:/milestone/all
svcadm milestone svc:/milestone/none
svcadm milestone help

svccfg

# create a new service
svccfg -h 
svccfg validate NewService.xml
svccfg import NewService.xml
svcadm enable NewService

services log location

  • each service has its own log
system-zones-monitoring:default.log
system-zones:default.log
oracle@enksc1db0201:/var/svc/log:dbm012
$ less system-pkgserv:default.log
oracle@enksc1db0201:/var/svc/log:dbm012
$ pwd
/var/svc/log




Administering Data Storage (ZFS)


ZFS does the following

  • storage
  • data integrity
  • encryption
  • backup and restore of files
  • creation and management of containers (zones)

ZFS features

  • enables addressing of multiple disk storage devices as a large contiguous block
  • 128-bit addressing (means no size restrictions of files)
  • 256-bit checksums on all disk operations
  • support RAID parity, striping, and mirroring schemes
  • automated detection and repair of corrupt data
  • encryption to protect sensitive data
  • data compression to save space
  • user storage quotas
  • sharing data with other ZFS pools
  • snapshot and recovery

ZFS terms

  • Filesystem
  • Pool - one or more disk devices or partitions
  • Clone - an exact copy of a ZFS filesystem
  • Snapshot - a copy of the state of the filesystem
  • Checksum - check integrity
  • Quota - limit on the storage amount for a user

ZFS storage pools

  • created and configured using the /usr/sbin/zpool command
  • the rpool is the default ZFS pool

zpool commands


CLI - create a pool

mkdir /zfstest
cd /zfstest
mkfile -n 100m testdisk1
mkfile -n 100m testdisk2
mkfile -n 100m testdisk3
mkfile -n 100m testdisk4

zpool create testpool /zfstest/testdisk1 /zfstest/testdisk2 /zfstest/testdisk3

root@enksc1db0201:/zfstest# zpool list
NAME      SIZE  ALLOC  FREE  CAP  DEDUP  HEALTH  ALTROOT
rpool     416G   172G  244G  41%  1.00x  ONLINE  -
testpool  285M   164K  285M   0%  1.00x  ONLINE  -

root@enksc1db0201:/zfstest# zpool status
  pool: rpool
 state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
        pool will no longer be accessible on older software versions.
  scan: none requested
config:

        NAME                         STATE     READ WRITE CKSUM
        rpool                        ONLINE       0     0     0
          mirror-0                   ONLINE       0     0     0
            c0t5000CCA01D8F2528d0s0  ONLINE       0     0     0
            c0t5000CCA01D8FC350d0s0  ONLINE       0     0     0

errors: No known data errors

  pool: testpool
 state: ONLINE
  scan: none requested
config:

        NAME                  STATE     READ WRITE CKSUM
        testpool              ONLINE       0     0     0
          /zfstest/testdisk1  ONLINE       0     0     0
          /zfstest/testdisk2  ONLINE       0     0     0
          /zfstest/testdisk3  ONLINE       0     0     0

errors: No known data errors



root@enksc1db0201:/zfstest# df -h
Filesystem             Size   Used  Available Capacity  Mounted on
testpool               253M    31K       253M     1%    /testpool




zpool destroy testpool


ZFS file systems

https://docs.oracle.com/cd/E23824_01/html/821-1448/gaynd.html

ZFS commands



CLI - create a filesystem

zfs create testpool/data



root@enksc1db0201:/zfstest# df -h
Filesystem             Size   Used  Available Capacity  Mounted on
testpool               253M    32K       253M     1%    /testpool
testpool/data          253M    31K       253M     1%    /testpool/data


mount/unmount

zfs unmount testpool/data
zfs mount testpool/data

# to mount all
zfs mount -a


ZFS snapshots and clones

snapshot

  • a snapshot is a read-only copy of the state of a ZFS filesystem
  • takes up almost no disk space
  • keeps track of only changes to the filesystem

clone

  • a clone is a writeable copy of a snapshot
  • used to turn a snapshot into a complete filesystem
  • can be placed within any other pool

administering ZFS snapshot and clone

  • Timeslider is a GUI tool to manage snapshots
  • you can also use zfs commands

create a snapshot

zfs snapshot <poolname>@<snapshotname>
zfs snapshot testpool@friday

# to rollback to a given snapshot
# you would usually rollback to the most recent snapshot, 
# if you need to rollback on older then you need to delete the most recent snapshot
zfs rollback 

list/get all snapshots

root@enksc1db0201:~# zfs get all | grep -i "type                             snapshot"
rpool/ROOT/SCMU_2016.07@install                  type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07@snapshot                 type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07@2016-08-30-04:02:48      type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07@2016-08-30-08:57:45      type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07@2016-08-30-20:29:07      type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07/var@install              type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07/var@snapshot             type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07/var@2016-08-30-04:02:48  type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07/var@2016-08-30-08:57:45  type                             snapshot                                         -
rpool/ROOT/SCMU_2016.07/var@2016-08-30-20:29:07  type                             snapshot                                         -
testpool@friday                                  type                             snapshot                                         -


create a clone

zfs clone



troubleshooting ZFS


get history of chages

zpool history

get info on pool and filesystem

zfs get all
zfs list
zpool status


Administering Oracle Solaris Zones


Zone configuration


Zone resource utilization


Administering zones


Zone and resource issues




Administering a Physical Network



Administering User Accounts



System and File Access



System Processes and Tasks











R forecasting step by step

$
0
0
    forecast step by step: 
        eyeball the data
            raw data    
            data exploration
            periodicity
            ndiff (how much we should difference)
            decomposition - determine the series components (trend, seasonality etc.)
                x = decompose(AirPassengers, "additive")
                mymodel = x$trend + x$seasonal; plot(mymodel)           # just the trend and seasonal data
                mymodel2 = AirPassengers - x$seasonal ; plot(mymodel2)  # orig data minus the seasonal data
            seasonplot 
        process data
            create xts object
            create a ts object from xts (coredata, index, frequency/periodicity)
            partition data train,validation sets        
        graph it 
            tsoutliers (outlier detection) , anomaly detection (AnomalyDetection package)
            log scale data
            add trend line (moving average (centered - ma and trailing - rollmean) and simple exponential smoothing (ets))
        performance evaluation
            Type of seasonality assessed graphically (decompose - additive,etc.)
            detrend and seasonal adjustment (smoothing/deseasonalizing)
            lag-1 diff graph
            forecast residual graph
            forecast error graph
            acf/pacf (Acf, tsdisplay)
                raw data
                forecast residual
                lag-1 diff
            autocorrelation 
                fUnitRoots::adfTest() - time series data is non-stationary (p value above 0.05)
                tsdisplay(diff(data_ts, lag=1)) - ACF displays there's no autocorrelation going on (no significant lags out of the 95% confidence interval, the blue line) 
            accuracy
            cross validation https://github.com/karlarao/forecast_examples/tree/master/cross_validation/cvts_tscvexample_investigation
            forecast of training
            forecast of training + validation + future (steps ahead)       
        forecast result
            display prediction intervals (forecast quantile)
            display the actual and forecasted series
            displaying the forecast errors
            distribution of forecast errors

facebook prophet

$
0
0

R stan bayesian

$
0
0
Stan for the beginners [Bayesian inference] in 6 mins (close captioned) https://www.youtube.com/watch?v=tLprFqSWS1w
A visual guide to Bayesian thinking https://www.youtube.com/watch?v=BrK7X_XlGB8
http://andrewgelman.com/2014/01/21/everything-need-know-bayesian-statistics-learned-eight-schools/
http://andrewgelman.com/2014/01/17/think-statistical-evidence-statistical-evidence-cant-conclusive/
An Introduction to Bayesian Inference using R Interfaces to Stan http://user2016.org/tutorials/15.html
http://andrewgelman.com/2012/08/30/a-stan-is-born/
http://mc-stan.org/interfaces/rstan.html
https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started
video https://icerm.brown.edu/video_archive/#/play/1107 Scalable Bayesian Inference with Hamiltonian Monte Carlo - Michael Betancourt, University of Warwick
Scalable Bayesian Inference with Hamiltonian Monte Carlo https://www.youtube.com/watch?v=VnNdhsm0rJQ
Efficient Bayesian inference with Hamiltonian Monte Carlo Michael Betancourt (Part 1) https://www.youtube.com/watch?v=pHsuIaPbNbY
Hamiltonian Monte Carlo and Stan Michael Betancourt (Part 2) https://www.youtube.com/watch?v=xWQpEAyI5s8
https://cran.r-project.org/web/packages/rstan/vignettes/rstan.html
https://rpubs.com/pviefers/CologneR
http://astrostatistics.psu.edu/su14/lectures/Daniel-Lee-Stan-1.pdf

others

http://stackoverflow.com/questions/31409591/difference-between-forecast-and-predict-function-in-r
http://stackoverflow.com/questions/28695076/parallel-predict
https://www.r-bloggers.com/parallel-r-model-prediction-building-and-analytics/

monte carlo simulation

$
0
0


Monte Carlo (mostly using probabilistic and permutation approaches and frameworks) , Markov Chain Monte Carlo , and Hamiltonian Monte Carlo. I think they all do the same thing with subtle difference, but it's confusing what to use and where to use it best ? What kind of Monte Carlos does this course teaches?

Facebook just released a forecasting package called "Prophet" and it uses Hamiltonian Monte Carlo https://research.fb.com/prophet-forecasting-at-scale/ using Stan http://mc-stan.org/ under the hood



monte carlo simulation forecasting in R
http://www.youtube.com/results?search_query=monte+carlo+simulation+forecasting+in+R

Prescriptive Analytics: Making Better Decisions with Simulation
http://www.b-eye-network.com/view/17224

Method for Creating Multipass Aggregations Using Tableau Server < doing various statistical methods in tableau
http://community.tableausoftware.com/message/181143#181143

monte carlo google search
https://www.google.com/search?q=what+is+monte+carlo+simulation&oq=what+is+monte+carlo+&aqs=chrome.1.69i57j0l5.4664j0j7&sourceid=chrome&es_sm=119&ie=UTF-8#q=monte+carlo+simulation+with+R

Introducing Monte Carlo Methods with R
http://www.stat.ufl.edu/archived/casella/ShortCourse/MCMC-UseR.pdf

Forecasting Hotel Arrivals and Occupancy Using Monte Carlo Simulation < good stuff, shows graph of actual vs forecast
http://alumnus.caltech.edu/~amir/hotelsim1.pdf

Why most sales forecasts suck…and how Monte Carlo simulations can make them better < monte carlo on sales forecast
http://www.retailshakennotstirred.com/retail-shaken-not-stirred/2010/01/why-most-sales-forecasts-suck-and-how-monte-carlo-simulations-can-make-them-better.html

Data Tables & Monte Carlo Simulations in Excel – A Comprehensive Guide < monte carlo in excel
http://chandoo.org/wp/2010/05/06/data-tables-monte-carlo-simulations-in-excel-a-comprehensive-guide/#montecarlo-simulations

Excel - Introduction to Monte Carlo simulation
http://office.microsoft.com/en-us/excel-help/introduction-to-monte-carlo-simulation-HA001111893.aspx

Monte Carlo in Tableau
http://drawingwithnumbers.artisart.org/basic-monte-carlo-simulations-in-tableau/

lynda.com - monte carlo simulation
http://www.lynda.com/search?q=monte+carlo+simulation

very easy explanation of monte carlo - lebron james example https://www.khanacademy.org/partner-content/lebron-asks-subject/lebron-asks/v/monte-carlo-simulation-to-answer-lebron-s-question here's the code https://www.khanacademy.org/cs/basketball-decisions/1024155511


stock trading references

http://jbmarwood.com/monte-carlo-analysis/
http://www.amazon.com/gp/product/0979183820/ref=as_li_qf_sp_asin_il_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0979183820&linkCode=as2&tag=marwmedi-20&linkId=7MNWGNX7KNQNCFXR
http://www.amazon.com/gp/product/B002DHMZ7O/ref=as_li_qf_sp_asin_il_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B002DHMZ7O&linkCode=as2&tag=marwmedi-20&linkId=KTT75L2DOWWVMMBC
https://www.youtube.com/results?search_query=monte+carlo+simulation+trading
https://www.youtube.com/watch?v=3gcLRU24-w0
https://www.linkedin.com/pulse/evaluating-your-strategy-tad-slaff<- Evaluating Your Strategy
https://www.linkedin.com/pulse/evaluating-your-strategy-part-2-tad-slaff


the best material so far

https://iversity.org/en/my/courses/monte-carlo-methods-in-finance/lesson_units#chapter101
http://www.quora.com/What-are-the-math-skills-you-need-for-statistical-arbitrage
http://www.quora.com/How-would-you-use-Monte-Carlo-simulation-to-predict-stock-movement
http://www.quora.com/What-specific-quantitative-trading-strategies-have-generated-the-most-aggregate-wealth-over-the-past-10-years
http://www.quora.com/What-are-some-of-the-most-important-algorithms-used-in-quantitative-trading




Markov Chain Monte-Carlo

(ML 18.1) Markov chain Monte Carlo (MCMC) introduction https://www.youtube.com/watch?v=12eZWG0Z5gY
https://www.youtube.com/watch?v=vTUwEu53uzs A Beginner's Guide to Monte Carlo Markov Chain MCMC Analysis 2016
https://www.youtube.com/watch?v=6BPArbP4bZohttps://www.youtube.com/watch?v=6BPArbP4bZo
https://www.safaribooksonline.com/library/view/learning-probabilistic-graphical/9781784392055/ Learning Probabilistic Graphical Models in R
https://www.safaribooksonline.com/library/view/learning-probabilistic-graphical/9781784392055/ch05s05.html Markov Chain Monte-Carlo





howto NFS on Solaris

$
0
0
how to add the following types of NFS mounts to each server (both app and database)
The mounts are specific to each server (exports on the ZFS has unique names).


Below are the Linux version of the mounts that we need help converting to Solaris:

# ZFS mounts (for SAP) - Environment specific (by application name/environment)
10.10.10.10:/export/sapmntCBP                        /sapmnt/CBP    nfs     rw,bg,hard,rsize=32768,wsize=32768,timeo=10,intr,proto=tcp,noac,vers=3,suid,_netdev 0 0
10.10.10.10:/export/ifaceCBP/nonprod          /iface/CBP          nfs     rw,bg,hard,rsize=32768,wsize=32768,timeo=10,intr,proto=tcp,noac,vers=3,suid,_netdev 0 0
10.10.10.10:/export/cbpci/usrsap                      /usr/sap               nfs     rw,bg,hard,rsize=32768,wsize=32768,timeo=10,intr,proto=tcp,noac,vers=3,suid,_netdev 0 0
10.10.10.10:/export/cbpci/oracle                      /oracle                  nfs     rw,bg,hard,rsize=32768,wsize=32768,timeo=10,intr,proto=tcp,noac,vers=3,suid,_netdev 0 0
10.10.10.10:/export/usrsaptrans_ce_np        /usr/sap/trans nfs     rw,bg,hard,rsize=32768,wsize=32768,timeo=10,intr,proto=tcp,noac,vers=3,suid,_netdev 0 0
# All SAP servers - For install only (then unmount)
10.243.204.253:/export/linux                       /sw            nfs rsize=8192,wsize=8192,timeo=14,intr,_netdev 0 0
10.10.10.10:/export/SAP_media         /SAP_media     nfs     rw,bg,hard,rsize=32768,wsize=32768,timeo=10,intr,proto=tcp,noac,vers=3,suid,_netdev 0 0           -
10.10.10.10:/export/Install                   /Install       nfs     rw,bg,hard,rsize=32768,wsize=32768,timeo=10,intr,proto=tcp,noac,vers=3,suid,_netdev 0 0


Solaris version

Generally the vfstab entry would be like this example:
#device           device   mount  FS    fsck   mount    mount
#to mount         to fsck  point  type  pass   at boot  options
IP:/export/file  -      /stage nfs   -     yes     rw,bg,hard,nointr,rsize=32768,wsize=32768,proto=tcp,noac,vers=3

The “suid" option is set by default but you can use it or “nosuid” if it needs to be off.  
There is no “_netdev" option, my understanding is in linux “_" options are ignored by the mount utility but are kept in the mount table as a flag apps can access with the mount command.  
I don’t know that I would use timeo=10 but if it is working for you now, try it. 
So your first entry below would look something like: 

10.10.10.10:/export/sapmntCBP - /sapmnt/CBP nfs – yes    rw,bg,hard,intr,rsize=32768,wsize=32768,timeo=10,proto=tcp,noac,vers=3,suid



other example

# Solaris DB exports
#10.10.14.10:/export/orainst - /oracleexp nfs - yes  rw,bg,hard,nointr,rsize=1048576,wsize=1048576,proto=tcp,suid,vers=3

# Linux DB exports  (used in ER2 for BOE (PBO, SBO and CBO)
10.10.14.10:/export/orainstx86 - /oracleexp nfs - yes  rw,bg,hard,nointr,rsize=1048576,wsize=1048576,proto=tcp,suid,vers=3







export the zone confiuration (to copy and create a new zone)

$
0
0
Zone root file systems are stored under:

/zones/<zone_name>

root@er1p2app02:/# cd /zones
root@er1p2app02:/zones# ls -al
total 53
drwxr-xr-x  17 root     root          17 Nov 10  2015 .
drwxr-xr-x  30 root     root          34 Feb  1 20:23 ..
drwx------   4 root     root           4 Jan 30 18:46 bw-zc
drwx------   4 root     root           4 Jan 30 18:46 ecc-zc
drwx------   4 root     root           4 Jan 30 18:47 er1zboe045v
drwx------   4 root     root           4 Jan 30 18:47 er1zboe049v
drwx------   4 root     root           4 Jan 30 18:47 er1zbw026v
drwx------   4 root     root           4 Jan 30 18:47 er1zbw030v
drwx------   4 root     root           4 Jan 30 18:47 er1zecc007v
drwx------   4 root     root           4 Jan 30 18:47 er1zecc010v
drwx------   4 root     root           4 Jan 30 18:47 er1zpo039v
drwx------   4 root     root           4 Jan 30 18:47 er1zpo041v
drwx------   4 root     root           4 Jan 30 18:47 er1zrhl014v
drwx------   4 root     root           4 Jan 30 18:46 grc-zc
drwx------   4 root     root           4 Jan 30 18:46 gts-zc
drwx------   4 root     root           4 Jan 30 18:46 po-zc
drwx------   4 root     root           4 Jan 30 18:46 rhl-zc
root@er1p2app02:/zones# cd bw-zc/
root@er1p2app02:/zones/bw-zc# ls -al
total 12
drwx------   4 root     root           4 Jan 30 18:46 .
drwxr-xr-x  17 root     root          17 Nov 10  2015 ..
drwxr-xr-x   2 root     root           2 Jan 30 18:46 lu
drwxr-xr-x  24 root     root          27 Feb 16 17:45 root

root@er1p2app02:~# zonecfg -z bw-zc export

create -b
set brand=solaris
set zonepath=/zones/bw-zc
set autoboot=false
set autoshutdown=shutdown
set limitpriv=default,proc_priocntl,proc_clock_highres
set ip-type=shared
add net
set address=er1zbw021v
set configure-allowed-address=true
set physical=sc_ipmp0
end
add net
set address=er1zbw021v-i
set configure-allowed-address=true
set physical=stor_ipmp0
end
add attr
set name=image-uuid
set type=string
set value=f844b7a5-153f-44fd-a0c2-562c89ef85f5
end
add attr
set name=cluster
set type=boolean
set value=true
end

Viewing all 1150 articles
Browse latest View live