Apache AGE (Incubating)

Apache AGE a PostgreSQL extension that provides graph database functionality. AGE is an acronym for A Graph Extension, and is inspired by Bitnine's fork of PostgreSQL 10, AgensGraph, which is a multi-model database. The goal of the project is to create single storage that can handle both relational and graph model data so that users can use standard ANSI SQL along with openCypher, the Graph query language.

Below is a brief overview of the AGE architecture in relation to the PostgreSQL architecture and backend. Every component runs on the PostgreSQL transaction cache layer and storage layer.

Features

AGE is currently being developed for the PostgreSQL 11 release and will support PostgreSQL 12 and 13 in 2021 and all the future releases of PostgreSQL.
AGE supports the openCypher graph query language and label hierarchy.
AGE enables querying multiple graphs at the same time. This will allow a user to query two or more graphs at once with cypher, decide how to merge them and get the desired query outputs.
AGE will be enhanced with an aim to support all of the key features of AgensGraph (PostgreSQL fork extended with graph DB functionality).

License

Apache AGE is licensed under Apache License, version 2.0

Roadmap

v0.1.0-alpha ✅
Release Date	2020 Q1
Release Notes	https://github.com/apache/incubator-age/releases/tag/v0.1.0
Key Improvements	Agtype to support all datatype requirements in Cypher queries Basic MATCH and CREATE clause support Expression support

v0.2.0-alpha ✅
Release Date	2020 Q2
Release Notes	https://github.com/apache/incubator-age/releases/tag/v0.2.0
Key Improvements	Extend Agtype to function better for SQL queries Scalar functions Advanced MATCH and CREATE clause support (except for VLE)

v0.3.0-incubating (official Apache incubator release) ✅
Release Date	2021-JAN-23
Release Notes	https://github.com/apache/incubator-age/releases/tag/v0.3.0.rc0-incubating
Key Improvements	Aggregation query support DELETE, SET, REMOVE, MERGE clause support Label Inheritance

v0.4.0-incubating (official Apache incubator release) ✅
Release Date	2021-APR-08
Release Notes	https://github.com/apache/incubator-age/releases/tag/0.4.0
Key Improvements	Aggregation support (Auto Group By, more input types) Basic agtype parser for the NodeJS driver Graph visualization tool integration Support for stored procedures and PL/pgSQL

v0.5.0-incubating
Release Date	Target 2021-JUL-05
Release Notes	To be posted
Key Improvements	Agtype's support for Hash Join for better performance Improved support for DELETE Support for CASE stetement and PostgreSQL EXPLAIN command ANTLR4 grammar file for JDBC, NodeJS drivers

v0.6.0-incubating
Release Date	Target 2021-JUL-30
Release Notes	To be posted
Key Improvements	openCypher VLE (Variable Length Edges) openCypher MERGE ...

* Support for PostgreSQL 12 will be implemented in Q3 2021.

Installing AGE

Using Docker

Docker images are available on Docker Hub and are based on the official PostgreSQL 11 Debian and Alpine images.

Installing from source

Building AGE from source depends on the following Linux libraries (Ubuntu package names shown below):

$ sudo apt-get install bison build-essential flex postgresql-server-dev-11

The build process will attempt to use the first path in the PATH environment variable when installing AGE. If the pg_config path if located there, run the following command in the source code directory of Apache AGE to build and install the extension.

$ make install

If the path to your Postgres installation is not in the PATH variable, add the path in the arguments:

$ make PG_CONFIG=/path/to/postgres/bin/pg_config install

Since Apache AGE will be installed in the directory of the PostgreSQL installation, proper permissions on the directory are required. Run the following statements in psql to create and load age in PostgreSQL.

=# CREATE EXTENSION age; -- run this statement only once CREATE EXTENSION =# LOAD 'age'; LOAD =# SET search_path = ag_catalog, "$user", public; SET

Getting Started

Loading AGE

Connect to your containerized Postgres instance and then run the following commands:

CREATE EXTENSION age; LOAD 'age'; SET search_path = ag_catalog, "$user", public;

Using AGE

First you will need to create a graph:

SELECT create_graph('my_graph_name');

To execute Cypher queries, you will need to wrap them in the following syntax:

SELECT * from cypher('my_graph_name', $$ CypherQuery $$) as (a agtype);

For example, if we wanted to create a graph with 4 nodes, we could do something as shown below:

SELECT * from cypher('my_graph_name', $$ CREATE (a:Part {part_num: '123'}), (b:Part {part_num: '345'}), (c:Part {part_num: '456'}), (d:Part {part_num: '789'}) $$) as (a agtype); --- RESULTS a --- (0 rows)

Then we could query the graph with the following:

SELECT * from cypher('my_graph_name', $$ MATCH (a) RETURN a $$) as (a agtype); --- RESULTS a ------------------------------------------------------------------------------------- {"id": 844424930131969, "label": "Part", "properties": {"part_num": "123"}}::vertex {"id": 844424930131970, "label": "Part", "properties": {"part_num": "345"}}::vertex {"id": 844424930131971, "label": "Part", "properties": {"part_num": "456"}}::vertex {"id": 844424930131972, "label": "Part", "properties": {"part_num": "789"}}::vertex (4 rows)

Next, we could create a relationship between a couple of nodes:

SELECT * from cypher('my_graph_name', $$ MATCH (a:Part {part_num: '123'}), (b:Part {part_num: '345'}) CREATE (a)-[u:used_by { quantity: 1 }]->(b) $$) as (a agtype); --- RESULTS a --- (0 rows)

Next we can return the path we just created (results have been formatted for readability):

SELECT * from cypher('age', $$ MATCH p=(a)-[]-(b) RETURN p $$) as (a agtype); // RESULTS // ROW 1 [ { "id":844424930131969, "label":"Part", "properties":{ "part_num":"123" } }::"vertex", { "id":1125899906842625, "label":"used_by", "end_id":844424930131970, "start_id":844424930131969, "properties":{ "quantity":1 } }::"edge", { "id":844424930131970, "label":"Part", "properties":{ "part_num":"345" } }::"vertex" ]::"path" // ROW 2 [ { "id":844424930131970, "label":"Part", "properties":{ "part_num":"345" } }::"vertex", { "id":1125899906842625, "label":"used_by", "end_id":844424930131970, "start_id":844424930131969, "properties":{ "quantity":1 } }::"edge", { "id":844424930131969, "label":"Part", "properties":{ "part_num":"123" } }::"vertex" ]::"path" (2 rows)

How to contribute

There are multiple ways you can contribute to the project. And help is always welcome! All details can be found on the contributing page. Keep reading for a quick overview!

Becoming a Committer

If you are interested in the project and looking for ways to help, consult the list of tasks in JIRA, or ask the mailing list.

Contributing by Helping Other Users

A great way to contribute to AGE is to help answer user questions on the mailing list or on StackOverflow. There are always many new AGE users; taking a few minutes to help answer a question is a very valuable community service.
Contributors should subscribe to this list and follow it to keep up to date on what's happening in AGE. Answering questions is an excellent and visible way to help the community, which also demonstrates your expertise.

Contributing by Reviewing Changes

Changes to AGE source code are proposed, reviewed, and committed via Github pull requests (described in Developer Guidelines). Anyone can view and comment on active changes here. Reviewing others' changes is a good way to learn how the change process works and gain exposure to activity in various parts of the code. You can help by reviewing the changes and asking questions or pointing out issues as simple as typos or small issues of style.

Contributing Documentation Changes

To propose a change to release documentation (that is, docs that appear here), edit the Markdown source files in AGE's doc/ directory on GitHub. Instructions for building the documentation can be found here. The process to propose a doc change is otherwise the same as the process for proposing code changes below.

Contributing Bug Reports

Ideally, bug reports are accompanied by a proposed code change to fix the bug. This isn't always possible, as those who discover a bug may not have the experience to fix it. A bug may be reported by creating a JIRA issue but without creating a pull request.
Bug reports are only useful, however, if they include enough information to understand, isolate and ideally reproduce the bug. Simply encountering an error does not mean a bug should be reported; search JIRA and inquire on the AGE user / dev mailing lists first. Unreproducible bugs, or simple error reports, may be closed.
The more context the reporter can give about a bug, the better, such as: how the bug was introduced, by which commit, etc. It assists the committers in the decision process on how far the bug fix should be backported, when the pull request is merged. The pull request to fix the bug should narrow down the problem to the root cause.
Data correctness/data loss bugs are very serious. Make sure the corresponding bug report JIRA issue is labeled as correctness or data-loss. Please send an email to dev@age.apache.org after submitting the bug report, to quickly draw attention to the issue.
Performance issues are classified as bugs. The pull request to fix a performance bug must provide a benchmark to prove the problem is indeed fixed.

Contributing to JIRA Maintenance

Most contributors can directly resolve JIRA issues. Use judgment in determining whether you are confident the issue should be resolved (although changes can be easily undone).
When resolving JIRA tickets, please observe the following conventions:

Resolve as Fixed if there's a release or code commit that resolved the issue.
- Set Fix Version(s), if and only if the resolution is Fixed
- Set Assignee to the person who contributed the most to its resolution, usually the person who opened the PR that resolved the issue.
For issues that can't be reproduced against master as reported, resolve as Cannot Reproduce.
If the issue is the same as or a subset of another issue, resolved as Duplicate
- Mark the issue that has less activity or discussion as the duplicate.
- Link it to the JIRA ticket it duplicates.
If the issue seems clearly obsolete and applies to issues or components that have changed radically since it was opened, resolve as Not a Problem
If the issue doesn't make sense or is not actionable resolve as Invalid.
If it's a coherent issue, but there is a clear indication that there is no support or interest in acting on it, then resolve as Won't Fix.

Searching JIRA How to search JIRA for existing issues:

Basic:
1. Visit AGE JIRA
2. Type query into search box in top right (beware this will return results from all projects)
Advanced:
1. Visit AGE JIRA Advanced Search
2. Replace the empty quotes with quoted text (such as "cypher")
Search Now:
1. Use the search box below

If a logically similar issue already exists, then contribute to the discussion on the existing JIRA ticket and pull request, instead of creating a new ticket and PR.

Developer Guidelines

Table of Contents

Code Review Process
How to Merge a Pull Request
Coding Style Guide
Creating Documentation

Code Review Process

Make a commit (or multiple commits) on your local branch.
Create .patch file(s) of the commit(s).
- Use git format-patch command.
Send the .patch file(s) to the reviewer.
- The title of the email must be "[Review] [JIRA Ticket Name Here] summary-of-the-issue"
  (e.g. [Review] [JIRA Ticket Name] Support changing graph name)
  - If the commit is not for any issues on JIRA, omit " [JIRA Ticket Name Here]". OR make a JIRA ticket
- The email body will look like this:
  Commit bef50e5d86d45707806f5733695a229f3e295b1a [one blank line] Description
  - The first line is the hash code of the base commit, NOT the commit you've created.
    - This will help reviewers to quickly apply the .patch files.
  - Put proper information to help the reviewer.
- Attach .patch files.
  - Do NOT rename files. They are named with numbers in order.
  - Do NOT compress them unless the total file size is over 5MB.
Reply to the last email in the same thread to send a review of it.
- You can attach some .diff files.
Reply to the last email in the same thread to send updated patch(es) and opinions.
- If you rebase commits, state the hash code of the new base commit.
Repeat 4 and 5.

How to Merge a Pull Request

Single Commit for a Single Task

In this case, the commit will be merged into the master branch with the following process.

Change the current working branch to the local master branch by running the following command.
$ git checkout master
Make the local master branch up-to-date by running the following command (or any other commands that result the same.)
$ git pull
Change the current working branch to the local task branch that the commit resides in by running the following command.
$ git checkout
Rebase the local task branch by running the following command.
$ git rebase master
Resolve any conflicts that occur during rebase.
Change the current working branch to the local master branch by running the following command.
$ git checkout master
Merge the local task branch into the local master branch by running the following command.
$ git merge

Multiple Commits for a Single Task

Keeping Commit History
Sometimes, having logically separated, multiple commits for a single task helps developers to grasp the logical process of the work that had been done for the task. If the commits are merged with fast-forward strategy, the commits will not be grouped together. Therefore, to group the commits, create an explicit merge commit.
In this case, the commits will be merged into the master branch with the same process above except the last step (step 7).
For the last step, the local task branch will be merged into the local master branch with an explicit merge commit by running the following command. If you omit --no-ff option, the command will do fast-forward merge instead.

$ git merge --no-ff

The above process will result, for example, the following commit history. (This is captured from apache/incubator-age.) There is an explicit merge commit, 69f3b32. Each explicit merge commit groups related commits.

* 9779a0b Implement property and element access operators * 69f3b32 Implement + (concatenating strings) operator |\ | * ab50b5c Support Floating Point Precision in String Operators | * cceebcd Implement String Operators |/ * 6c36b80 Fix failed assertion when agtype_build_map() takes agtype as key * 304bc68 Refactor bool_to_agtype()

Note: There is no commit between an explicit merge commit and the parent commit, which is on the master branch, of the explicit merge commit. This is done by doing rebase before merge.

Code Style Guide

For a full list of coding style guidelines, please refer to the style setup in the clang-format.5 in the AGE git repository.

Indentation

Use 4 spaces per indentation level. (no tab character)

You can see the same indentation in all environments.

For switch statement, see below.

switch (suffix) { case 'G': case 'g': mem <<= 30; break; case 'M': case 'm': mem <<= 20; break; case 'K': case 'k': mem <<= 10; // fall through default: break; }

Breaking long lines and strings

The line length limit is 79 columns, except for strings longer than 79 characters.

Placing Braces and Spaces

All braces are on their own line solely. See below.

int function(int x) { body of function } struct s { int a; int b; } if (x is true) { we do a } else if (y is true) { we do b } else { we do c we do d }

If all the bodies of if/else statement contain a single line, omit braces. See below.

if (x is true) we do a if (y is true) we do b else we do c

One exception is do statement. See the following example.

do { body of do-loop } while (condition);

Naming

Use the underscore name convention for all variables, functions, structs, enums and define macros.

Typedefs

Use typedef only for struct and enum. It must not be used for pointer types.

Commenting

For multi-line comments, use C style multi-line comments.
For single-line comments, use C++ style single-line comments.
See below.

/* * This function * does x */ void f(void) { // This is to check... if (y is true) we do b /* * We need to do this here * because of ... */ for (;;) }

Macros, Enums, and RTL

Don't align bodies of macros.

If names are changed or new entries are added, it may produces unnecessary diffs and this makes it harder for you to find lines that are actually modified

//do this #define Anum_ag_graph_name 1 #define Anum_ag_graph_namespace 2 //not this #define Anum_ag_graph_name 1 #define Anum_ag_graph_namespace 2

When you write a macro that spans multiple lines, don't align \ character.

If bodies of macros are modified, it may produce unnecessary diffs and this makes it harder for you to find lines that are actually modified.
It is harder for you to find lines that are actually modified.

// do this #define f() \ do \ { \ run(); \ } while (0) // not this #define f() \ do \ { \ run(); \ } while (0)

Newlines

For newlines, only \n is allowed, not \r\n and \r.

Conditions

If a pointer variable (including List *) is used as a condition, which means that it is evaluated as true/false value, use it AS-IS. Do not perform explicit comparison with NULL (or NIL). For negation, put ! before it.

Rules for ereport()

An error message that is passed to errmsg() starts with a lower case letter.
An error detail/hint message that is passed to errdetail()/errhint() starts with an upper case letter.

Creating Documentation

Dependencies: python3, python3-sphinx, sphinx_rtd_theme
Clone incubator-age repository (AGE) (https://github.com/apache/incubator-age)
Go to the doc folder of the project
Type make html
The documents are in _build/html

An example of the commands on Ubuntu follow:

sudo apt install python3 python3-sphinx pip3 install sphinx_rtd_theme git clone https://github.com/apache/incubator-age cd incubator-age/doc make html

Mailing Lists

Get help using Apache AGE (Incubating) or contribute to the project on our mailing lists:

Visit Commits or subscribe for people following commits to the project.
Visit Developer or subscribe for developers wanting to commit code to the project.
Visit Users or subscribe for users wanting to learn more about the project.

Apache AGE Team Members

✪ Mentors ✪

AGE's Mentors are (sorted on last name in alphabetical order):

Name	Email
Raphael Bircher	rbircher at apache dot org
Von Gosling	vongosling at apachedot org
Jim Jagielski	jim at apache dot org
Kevin Ratnasekera	kevin at agsft dot com
Felix Cheung

✰ Committers ✰

AGE's active Committers are (sorted on last name in alphabetical order):

Name	Email
John Gemignani	john dot gemignani at bitnine dot net
Josh Innis	josh dot innis at bitnine dot net
Eya Badal	eya dot abdisho at bitnine dot net

✧ Contributors ✧

AGE's Contributors are (sorted on last name in alphabetical order):

Name	Email
Ghassan Al-Noubani	g.noubani at qbs dot jo
Jasper Blues	jasper at liberation dash data dot com
Dehowe Feng
Donghu Kim	donghu dot kim at oracle dot com
Andrew Ko	andrew dot ko at bitnine dot net
Mason Sharp	mason dot sharp at gmail dot com
Muhammad Shoaib	Muhammad dot Shoaib at uni dot lu
Nick Sorrell	nick at canvouch dot com
Junseok Yang