wtorek, 31 maja 2022

Async Profiler - quick profiling for any application

Are you tired of setting up Java APM agents, configuring some arcane ENV variables (does "org.jboss.logmanager" ring any bells)? Just try Async Profiler ( https://github.com/jvm-profiling-tools/async-profiler ). It is even trivial to use on your already *running* docker images. For example: 


$ kubectl -n my-namespace cp ./async-profiler-2.8-linux-x64.tar.gz my-pod:/root $ winpty kubectl -n my-namespace exec -it my-pod bash root@my-pod:/# cd /root root@my-pod:~# tar xf async-profiler-2.8-linux-x64.tar.gz root@my-pod:~# cd async-profiler-2.8-linux-x64 root@my-pod:~/async-profiler-2.8-linux-x64# ./profiler.sh -d 1200 -i 10ms -f profile.html 6 Profiling for 1200 seconds Done root@@my-pod:~/async-profiler-2.8-linux-x64# exit kubectl -n my-namespace cp my-pod:/root/async-profiler-2.8-linux-x64/profile.html ./profile.html
Where "-i 10ms" was how often to run sampling and "6" was the pid of java process (you can check for one with `jps` or plain old `ps`). You can view the resulting flame graph (the local "profile.html" file) in any browser.

poniedziałek, 20 kwietnia 2020

Sensible Hibernate batch properties


  <property name="hibernate.jdbc.batch_size" value="30"/>
  <property name="hibernate.order_inserts" value="true"/>
  <property name="hibernate.order_updates" value="true"/>
  <property name="hibernate.jdbc.batch_versioned_data" value="true"/>
Also for postgresql and hibernate < 5.2 add "?reWriteBatchedInserts=true" to connection url.

poniedziałek, 30 marca 2020

Checking private (non shared) memory usage for Postgresql connection (PID)

Create function:

 CREATE OR REPLACE FUNCTION pid_mem_usage( pid integer)
 RETURNS integer AS $$
 DECLARE l_command TEXT; res integer;
 BEGIN
  SET client_min_messages = 'error';
  l_command  := ' cat /proc/' || pid || '/smaps | grep "^Private" | awk ''{a+=$2}END{print a * 1024}'' ';
  raise notice '%',l_command;
  execute 'create temp table if not exists z (a int) ';
  execute 'copy z from program ' || quote_literal(l_command);
  execute 'select sum(a) from z' into res;
  execute 'truncate z';
 RETURN res;
 END;
 $$ LANGUAGE plpgsql;
Sample usage:
select pid_mem_usage(pid),pid,usename,backend_start,xact_start,
query from pg_stat_activity  order by 1 desc limit 25;

Based on from https://www.depesz.com/2012/06/09/how-much-ram-is-postgresql-using/

czwartek, 18 lipca 2019

Checking for index bloat in Psotgresql

SELECT
    t.tablename,
    indexname,
    c.reltuples AS num_rows,
    100.0*pg_relation_size(quote_ident(t.schemaname)::text||'.'||quote_ident(indexrelname)::text)/(pg_relation_size(quote_ident(t.schemaname)::text||'.'||quote_ident(t.tablename)::text)+1)as ratio,
    pg_relation_size(quote_ident(t.schemaname)::text||'.'||quote_ident(t.tablename)::text) AS table_size_raw,
    pg_relation_size(quote_ident(t.schemaname)::text||'.'||quote_ident(indexrelname)::text) AS index_size_raw,
    pg_size_pretty(pg_relation_size(quote_ident(t.schemaname)::text||'.'||quote_ident(t.tablename)::text)) AS table_size,
    pg_size_pretty(pg_relation_size(quote_ident(t.schemaname)::text||'.'||quote_ident(indexrelname)::text)) AS index_size,
    CASE WHEN indisunique THEN 'Y'
       ELSE 'N'
    END AS UNIQUE,
    idx_scan AS number_of_scans,
    idx_tup_read AS tuples_read,
    idx_tup_fetch AS tuples_fetched
FROM pg_tables t
LEFT OUTER JOIN pg_class c ON t.tablename=c.relname
LEFT OUTER JOIN
    ( SELECT c.relname AS ctablename, ipg.relname AS indexname, x.indnatts AS number_of_columns, idx_scan, idx_tup_read, idx_tup_fetch, indexrelname, indisunique FROM pg_index x
           JOIN pg_class c ON c.oid = x.indrelid
           JOIN pg_class ipg ON ipg.oid = x.indexrelid
           JOIN pg_stat_all_indexes psai ON x.indexrelid = psai.indexrelid AND (psai.schemaname='public' or psai.schemaname='partitions') )
    AS foo
    ON t.tablename = foo.ctablename
WHERE (t.schemaname='public' or t.schemaname='partitions') 

ORDER BY 5 desc, 6 desc
Compare ratio among partitions of same table to find bloated indexes - the one for current month is always bloated :).

wtorek, 28 maja 2019

jGitFlow: "CheckoutConflictException: Checkout conflict with files"

This error probably means the master and develop have diverged and develop is behind master. Just merge master to develop and try building again.

wtorek, 16 stycznia 2018

Wildfly: NoSuchMethodError: Logger.tracef

When adding Arquillian based integration tests to older projects I often encounter exception:
Exception in thread "Remoting "endpoint" task-4" java.lang.NoSuchMethodError: org.jboss.logging.Logger.tracef(Ljava/lang/String;I)V
        at org.jboss.remotingjmx.VersionedConectionFactory$ClientVersionReceiver.handleMessage(VersionedConectionFactory.java:158)
        at org.jboss.remoting3.remote.RemoteConnectionChannel$5.run(RemoteConnectionChannel.java:456)
        at org.jboss.remoting3.EndpointImpl$TrackingExecutor$1.run(EndpointImpl.java:717)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
It is caused a mismatch in versions of jboss-logging. The solution is to find additional version:
$ mvn dependency:tree |grep jboss-logging
...

[INFO] +- com.google.guava:guava:jar:18.0:compile
[INFO] +- org.jboss.resteasy:resteasy-client:jar:3.0.19.Final:compile
[INFO] |  +- org.jboss.resteasy:resteasy-jaxrs:jar:3.0.19.Final:compile
[INFO] |  |  +- org.jboss.spec.javax.ws.rs:jboss-jaxrs-api_2.0_spec:jar:1.0.0.Final:compile
[INFO] |  |  +- org.jboss.spec.javax.annotation:jboss-annotations-api_1.2_spec:jar:1.0.0.Final:compile
[INFO] |  |  +- javax.activation:activation:jar:1.1.1:compile
[INFO] |  |  +- commons-io:commons-io:jar:2.1:compile
[INFO] |  |  \- net.jcip:jcip-annotations:jar:1.0:compile
[INFO] |  \- org.jboss.logging:jboss-logging:jar:3.1.4.GA:compile
And to exclude it in pom.xml:
        <dependency>
            <groupId>org.jboss.resteasy</groupId>
            <artifactId>resteasy-client</artifactId>
            <version>3.0.19.Final</version>
            <scope>provided</scope>
            <type>jar</type>
            <exclusions>
                <exclusion>
                    <groupId>org.jboss.logging</groupId>
                    <artifactId>jboss-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

czwartek, 2 czerwca 2016

F5 VPN Client under Linux

It looks like many places have outdated or plain wrong instructions about connecting to f5 VPN under Linux is a bit outdated so a quick reminder:
  1. Ignore web plugin/extension, it probably won't work anyway under 64 bit browser with disabled NPAPI support
  2. download and extract https://vpn.somehost.com/public/download/linux_sslvpn.tgz
  3. execute Install.sh
  4. f5fpc --start -t vpn.somehost.com