Initial commit

This commit is contained in:
fg-admin 2023-01-25 13:40:44 +03:00
commit 369ffc8be8
61 changed files with 6979 additions and 0 deletions

4
.gitignore vendored Normal file
View File

@ -0,0 +1,4 @@
/qdsl
/cmd/qdsl/qdsl
/cmdb-agent
/cmd/cmdb-agent/cmdb-agent

34
.goreleaser.yml Normal file
View File

@ -0,0 +1,34 @@
project_name: cmdb
gitea_urls:
api: http://git.fg-tech.ru/api/v1
download: http://git.fg-tech.ru
skip_tls_verify: true
builds:
- main: ./cmd/qdsl
id: qdsl
binary: qdsl
goos:
- linux
goarch:
- amd64
ldflags:
- -X main.version={{ .Version }}
- -X main.release={{ .ShortCommit }}
env:
- CGO_ENABLED=0
nfpms:
- id: qdsl
file_name_template: "qdsl-{{ .Version }}-{{ .Os }}-{{ .Arch }}"
package_name: qdsl
maintainer: listware
description: QDSL
license: "Apache 2.0"
bindir: /usr/bin
builds:
- qdsl
formats:
- rpm
- deb
contents: []

202
LICENSE Normal file
View File

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

3
README.md Normal file
View File

@ -0,0 +1,3 @@
# Cluster Management Data Base [CMDB](./cmd/cmdb-agent)
# Query Database Search Language [QDSL](./cmd/qdsl)

9
cmd/cmdb-agent/README.md Normal file
View File

@ -0,0 +1,9 @@
# Cluster Management Data Base (CMDB)
## Stack
* [ArangoDB](https://github.com/arangodb/arangodb)
* [cmdb-agent](./cmd/cmdb-agent/main.go)
###### Feature
* All CRUD are performed on the graph
* Removing objects also destroys object links

11
cmd/cmdb-agent/main.go Normal file
View File

@ -0,0 +1,11 @@
// Copyright 2022 Listware
package main
import (
"git.fg-tech.ru/listware/cmdb/internal/server"
)
func main() {
server.New()
}

19
cmd/qdsl/README.md Normal file
View File

@ -0,0 +1,19 @@
# Query Database Search Language
* search and remove objects/links by qdsl query
###### Example
```
qdsl *.root
qdsl --remove --id *.root
qdsl --remove --linkid *.root
```
###### Flags
* `id - get vertex '_id', default true`
* `key - get vertex '_key', default false`
* `object - get vertex 'object', default false`
* `link - get edge 'object', default false`
* `linkId - get edge 'id', default false`
* `name - get edge '_name', default false`
* `type - get edge '_type', default false`
* `remove - remove all results, default false`
* `confirm - confirm remove, default false`

7
cmd/qdsl/main.go Normal file
View File

@ -0,0 +1,7 @@
// Copyright 2022 Listware
package main
func main() {
execute()
}

85
cmd/qdsl/qdsl.go Normal file
View File

@ -0,0 +1,85 @@
// Copyright 2022 Listware
package main
import (
"context"
"encoding/json"
"fmt"
"regexp"
"strings"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/qdsl"
"github.com/manifoldco/promptui"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
var options = qdsl.NewOptions()
var byPT string
var customFilter []string
var confirm bool
func buildFilter(query string, filters []string) string {
if len(filters) == 0 {
return query
}
// find *. or <. at the start
r, err := regexp.Compile(`^[\*\<]\.`)
if err != nil {
log.Error("Can't add filter, parse error: ", err)
return query
}
num := r.FindStringIndex(query)
if num != nil {
i := num[1] - 1
query = query[:i] + "[?" + filters[0] + "?]" + query[i:]
filters = filters[1:]
}
if strings.Index(query, "[?") != -1 {
i := strings.Index(query, "?]")
if i == -1 {
log.Error("Error while parsing filter: can't find close filter operator '?]'")
return query
}
newQuery := query[:i]
for _, filter := range filters {
newQuery += " && " + filter
}
newQuery += query[i:]
query = newQuery
// return newQuery
}
log.WithFields(logrus.Fields{"cli": "qdsl"}).Debug(query)
return query
}
func qdslQuery(cmd *cobra.Command, args []string) {
log.WithFields(logrus.Fields{"cli": "qdsl"}).Debug("QDSL called with argument: ", args[0])
query := buildFilter(args[0], customFilter)
if options.Remove && !confirm {
prompt := promptui.Prompt{
Label: fmt.Sprintf("Confirm remove %s", query),
IsConfirm: true,
}
_, err := prompt.Run()
if err != nil {
return
}
}
elements, err := qdsl.RawQdsl(context.Background(), query, options)
if err != nil {
log.Error(err)
return
}
s, _ := json.Marshal(elements)
fmt.Println(string(s))
}

76
cmd/qdsl/root.go Normal file
View File

@ -0,0 +1,76 @@
// Copyright 2022 Listware
package main
import (
"errors"
"fmt"
"os"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
var log = logrus.New()
var (
version = "v1.0.0"
release = "dev"
versionTemplate = `{{printf "%s" .Short}}
{{printf "Version: %s" .Version}}
Release: ` + release + `
`
)
// rootCmd represents the base command when called without any subcommands
var rootCmd = &cobra.Command{
Version: version,
Use: "qdsl QUERY",
Short: "CMDB query language",
Long: `CMDB query language for getting information about nodes`,
Args: func(cmd *cobra.Command, args []string) error {
if len(args) < 1 {
return errors.New("requires at least one arg")
}
return nil
},
Run: qdslQuery,
}
// Execute adds all child commands to the root command and sets flags appropriately.
// This is called by main.main(). It only needs to happen once to the rootCmd.
func execute() {
if err := rootCmd.Execute(); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
func init() {
rootCmd.SetVersionTemplate(versionTemplate)
// Add commands
rootCmd.AddCommand(autoShellCmd)
rootCmd.Flags().BoolVarP(&options.Key, "key", "k", false, "add key to result")
rootCmd.Flags().BoolVarP(&options.Id, "id", "i", false, "add id to result")
rootCmd.Flags().BoolVarP(&options.Type, "type", "t", false, `add type to result`)
rootCmd.Flags().BoolVarP(&options.Object, "object", "o", false, "add object to result")
rootCmd.Flags().BoolVarP(&options.Link, "link", "l", false, `add link to result`)
rootCmd.Flags().BoolVarP(&options.LinkId, "linkid", "I", false, "add link id to result")
rootCmd.Flags().BoolVarP(&options.Name, "name", "n", false, "add name in particular topology to result")
rootCmd.Flags().BoolVarP(&options.Path, "path", "p", false, `add path to result`)
rootCmd.Flags().BoolVarP(&options.Remove, "remove", "r", false, "remove result")
rootCmd.Flags().BoolVarP(&confirm, "confirm", "y", false, "confirm remove")
}
var autoShellCmd = &cobra.Command{
Use: "autoshell",
Short: "Generate bash completion script",
Long: "Generate bash completion script",
Hidden: true,
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Root().GenBashCompletionFile("/etc/bash_completion.d/qdsl.sh")
},
}

5
genpeg.go Normal file
View File

@ -0,0 +1,5 @@
// Copyright 2022 Listware
package genpeg
//go:generate pigeon -o ./internal/cmdb/qdsl/qdslpeg.go qdsl.peg

27
go.mod Normal file
View File

@ -0,0 +1,27 @@
module git.fg-tech.ru/listware/cmdb
go 1.19
require (
git.fg-tech.ru/listware/proto v0.1.1
github.com/arangodb/go-driver v1.4.1
github.com/bbuck/go-lexer v1.0.0
github.com/manifoldco/promptui v0.9.0
github.com/sirupsen/logrus v1.9.0
github.com/spf13/cobra v1.6.1
google.golang.org/grpc v1.52.1
)
require (
github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e // indirect
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/spf13/pflag v1.0.5 // indirect
golang.org/x/net v0.4.0 // indirect
golang.org/x/sys v0.3.0 // indirect
golang.org/x/text v0.5.0 // indirect
google.golang.org/genproto v0.0.0-20221118155620-16455021b5e6 // indirect
google.golang.org/protobuf v1.28.1 // indirect
)

64
go.sum Normal file
View File

@ -0,0 +1,64 @@
git.fg-tech.ru/listware/proto v0.1.1 h1:CSqteAtgysiJe7+KtLOEXSIvxypmlJCKwQtla1d2v+A=
git.fg-tech.ru/listware/proto v0.1.1/go.mod h1:t5lyMTuX/if05HI/na9tJAlHCHHMdhdPLBTkhvscedQ=
github.com/arangodb/go-driver v1.4.1 h1:Jg0N7XKxiKwjswmAcMCnefWmt81KJEqybqRAGJDRWlo=
github.com/arangodb/go-driver v1.4.1/go.mod h1:UTtaxTUMmyPWzKc2dsWWOZzZ3yM6aHWxn/eubGa3YmQ=
github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e h1:Xg+hGrY2LcQBbxd0ZFdbGSyRKTYMZCfBbw/pMJFOk1g=
github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e/go.mod h1:mq7Shfa/CaixoDxiyAAc5jZ6CVBAyPaNQCGS7mkj4Ho=
github.com/bbuck/go-lexer v1.0.0 h1:ZwzxWHnxQslJ/5I01nlhZwE7MtmvXbtEyRwrgQ70Qew=
github.com/bbuck/go-lexer v1.0.0/go.mod h1:JOt4Q0nNqWxYEy+spld4SJGe9r8G8suXd1dukur9O90=
github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/manifoldco/promptui v0.9.0 h1:3V4HzJk1TtXW1MTZMP7mdlwbBpIinw3HztaIlYthEiA=
github.com/manifoldco/promptui v0.9.0/go.mod h1:ka04sppxSGFAtxX0qhlYQjISsg9mR4GWtQEhdbn6Pgg=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA=
github.com/spf13/cobra v1.6.1/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
golang.org/x/net v0.4.0 h1:Q5QPcMlvfxFTAPV0+07Xz/MpK9NTXu2VDUuy0FeMfaU=
golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM=
golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/genproto v0.0.0-20221118155620-16455021b5e6 h1:a2S6M0+660BgMNl++4JPlcAO/CjkqYItDEZwkoDQK7c=
google.golang.org/genproto v0.0.0-20221118155620-16455021b5e6/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg=
google.golang.org/grpc v1.52.1 h1:2NpOPk5g5Xtb0qebIEs7hNIa++PdtZLo2AQUpc1YnSU=
google.golang.org/grpc v1.52.1/go.mod h1:pu6fVzoFb+NBYNAvQL08ic+lvB2IojljRYuun5vorUY=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@ -0,0 +1,70 @@
// Copyright 2022 Listware
package arangodb
import (
"context"
"crypto/tls"
"net/http"
"os"
driver "github.com/arangodb/go-driver"
arangohttp "github.com/arangodb/go-driver/http"
)
const (
cmdbName = "CMDB"
systemGraphName = "system"
)
var (
arangoAddr string
arangoUser string
arangoPassword string
)
func init() {
if value, ok := os.LookupEnv("ARANGO_ADDR"); ok {
arangoAddr = value
}
if value, ok := os.LookupEnv("ARANGO_USER"); ok {
arangoUser = value
}
if value, ok := os.LookupEnv("ARANGO_PASSWORD"); ok {
arangoPassword = value
}
}
func Connect() (client driver.Client, err error) {
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
// Open a client connection
conn, err := arangohttp.NewConnection(arangohttp.ConnectionConfig{
Transport: tr,
Endpoints: []string{arangoAddr},
})
if err != nil {
return
}
return driver.NewClient(driver.ClientConfig{
Connection: conn,
Authentication: driver.BasicAuthentication(arangoUser, arangoPassword),
})
}
func Database(ctx context.Context, client driver.Client) (driver.Database, error) {
return client.Database(ctx, cmdbName)
}
func Graph(ctx context.Context, client driver.Client) (graph driver.Graph, err error) {
db, err := client.Database(ctx, cmdbName)
if err != nil {
return
}
return db.Graph(ctx, systemGraphName)
}

View File

@ -0,0 +1,128 @@
// Copyright 2022 Listware
package arangodb
import (
"context"
driver "github.com/arangodb/go-driver"
)
const (
systemCollection = "system"
typesCollection = "types"
objectsCollection = "objects"
linksCollection = "links"
)
var (
allowUserKeysPtr = true
)
func Bootstrap(ctx context.Context) (err error) {
client, err := Connect()
if err != nil {
return
}
ok, err := client.DatabaseExists(ctx, cmdbName)
if err != nil {
return
}
if !ok {
options := &driver.CreateDatabaseOptions{}
if _, err = client.CreateDatabase(ctx, cmdbName, options); err != nil {
return
}
}
db, err := client.Database(ctx, cmdbName)
if err != nil {
return
}
// system collection
if ok, err = db.CollectionExists(ctx, systemCollection); err != nil {
return
}
if !ok {
options := &driver.CreateCollectionOptions{
IsSystem: true,
KeyOptions: &driver.CollectionKeyOptions{
AllowUserKeysPtr: &allowUserKeysPtr,
},
}
if _, err = db.CreateCollection(ctx, systemCollection, options); err != nil {
return
}
}
// types collection
if ok, err = db.CollectionExists(ctx, typesCollection); err != nil {
return
}
if !ok {
options := &driver.CreateCollectionOptions{
KeyOptions: &driver.CollectionKeyOptions{
AllowUserKeysPtr: &allowUserKeysPtr,
},
}
if _, err = db.CreateCollection(ctx, typesCollection, options); err != nil {
return
}
}
// objects collection
if ok, err = db.CollectionExists(ctx, objectsCollection); err != nil {
return
}
if !ok {
options := &driver.CreateCollectionOptions{
KeyOptions: &driver.CollectionKeyOptions{
Type: driver.KeyGeneratorTraditional,
},
}
if _, err = db.CreateCollection(ctx, objectsCollection, options); err != nil {
return
}
}
// links collection
if ok, err = db.CollectionExists(ctx, linksCollection); err != nil {
return
}
if !ok {
options := &driver.CreateCollectionOptions{
Type: driver.CollectionTypeEdge,
}
if _, err = db.CreateCollection(ctx, linksCollection, options); err != nil {
return
}
}
// system graph
if ok, err = db.GraphExists(ctx, systemGraphName); err != nil {
return
}
if !ok {
options := &driver.CreateGraphOptions{
EdgeDefinitions: []driver.EdgeDefinition{
driver.EdgeDefinition{
Collection: linksCollection,
From: []string{systemCollection, typesCollection, objectsCollection},
To: []string{typesCollection, objectsCollection},
},
},
}
if _, err = db.CreateGraphV2(ctx, systemGraphName, options); err != nil {
return
}
}
return
}

View File

@ -0,0 +1,38 @@
// Copyright 2022 Listware
package edge
import (
"context"
"encoding/json"
"fmt"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Create(ctx context.Context, client driver.Client, name string, payload any) (meta driver.DocumentMeta, resp map[string]any, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, _, err := graph.EdgeCollection(ctx, name)
if err != nil {
return
}
ctx = driver.WithReturnNew(ctx, &resp)
if b, ok := payload.([]byte); ok {
var req map[string]any
if err = json.Unmarshal(b, &req); err != nil {
return
}
fmt.Println("create edge bytes ", req)
meta, err = collection.CreateDocument(ctx, req)
return
}
meta, err = collection.CreateDocument(ctx, payload)
return
}

View File

@ -0,0 +1,23 @@
// Copyright 2022 Listware
package edge
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Read(ctx context.Context, client driver.Client, name, key string) (meta driver.DocumentMeta, resp map[string]any, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, _, err := graph.EdgeCollection(ctx, name)
if err != nil {
return
}
meta, err = collection.ReadDocument(ctx, key, &resp)
return
}

View File

@ -0,0 +1,22 @@
// Copyright 2022 Listware
package edge
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Remove(ctx context.Context, client driver.Client, name, key string) (meta driver.DocumentMeta, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, _, err := graph.EdgeCollection(ctx, name)
if err != nil {
return
}
return collection.RemoveDocument(ctx, key)
}

View File

@ -0,0 +1,33 @@
// Copyright 2022 Listware
package edge
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Update(ctx context.Context, client driver.Client, name, key string, payload any) (meta driver.DocumentMeta, resp map[string]any, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, _, err := graph.EdgeCollection(ctx, name)
if err != nil {
return
}
ctx = driver.WithReturnNew(ctx, &resp)
if b, ok := payload.([]byte); ok {
var req map[string]any
if err = json.Unmarshal(b, &req); err != nil {
return
}
meta, err = collection.ReplaceDocument(ctx, key, req)
return
}
meta, err = collection.ReplaceDocument(ctx, key, payload)
return
}

View File

@ -0,0 +1,35 @@
// Copyright 2022 Listware
package query
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Query(ctx context.Context, client driver.Client, query string, vars map[string]any) (metas []driver.DocumentMeta, resp []map[string]any, err error) {
db, err := arangodb.Database(ctx, client)
if err != nil {
return
}
cursor, err := db.Query(ctx, query, vars)
if err != nil {
return
}
defer cursor.Close()
for cursor.HasMore() {
var obj map[string]any
meta, err := cursor.ReadDocument(ctx, &obj)
if err != nil {
return nil, nil, err
}
resp = append(resp, obj)
metas = append(metas, meta)
}
return
}

View File

@ -0,0 +1,37 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Create(ctx context.Context, client driver.Client, name string, payload any) (meta driver.DocumentMeta, resp map[string]any, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, err := graph.VertexCollection(ctx, name)
if err != nil {
return
}
ctx = driver.WithReturnNew(ctx, &resp)
if b, ok := payload.([]byte); ok {
var req map[string]any
if err = json.Unmarshal(b, &req); err != nil {
return
}
meta, err = collection.CreateDocument(ctx, req)
return
}
meta, err = collection.CreateDocument(ctx, payload)
return
}

View File

@ -0,0 +1,24 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Read(ctx context.Context, client driver.Client, name, key string) (meta driver.DocumentMeta, resp map[string]any, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, err := graph.VertexCollection(ctx, name)
if err != nil {
return
}
meta, err = collection.ReadDocument(ctx, key, &resp)
return
}

View File

@ -0,0 +1,22 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Remove(ctx context.Context, client driver.Client, name, key string) (meta driver.DocumentMeta, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, err := graph.VertexCollection(ctx, name)
if err != nil {
return
}
return collection.RemoveDocument(ctx, key)
}

View File

@ -0,0 +1,33 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
driver "github.com/arangodb/go-driver"
)
func Update(ctx context.Context, client driver.Client, name, key string, payload any) (meta driver.DocumentMeta, resp map[string]any, err error) {
graph, err := arangodb.Graph(ctx, client)
if err != nil {
return
}
collection, err := graph.VertexCollection(ctx, name)
if err != nil {
return
}
ctx = driver.WithReturnNew(ctx, &resp)
if b, ok := payload.([]byte); ok {
var req map[string]any
if err = json.Unmarshal(b, &req); err != nil {
return
}
meta, err = collection.ReplaceDocument(ctx, key, req)
return
}
meta, err = collection.ReplaceDocument(ctx, key, payload)
return
}

View File

@ -0,0 +1,23 @@
// Copyright 2022 Listware
package edge
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
driver "github.com/arangodb/go-driver"
)
type Server struct {
pbcmdb.UnimplementedEdgeServiceServer
client driver.Client
}
func New(ctx context.Context) (s *Server, err error) {
s = &Server{}
s.client, err = arangodb.Connect()
return
}

View File

@ -0,0 +1,26 @@
// Copyright 2022 Listware
package edge
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/internal/arangodb/edge"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Create(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, resp, err := edge.Create(ctx, s.client, request.GetCollection(), request.GetPayload())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
response.Payload, err = json.Marshal(resp)
return
}

View File

@ -0,0 +1,27 @@
// Copyright 2022 Listware
package edge
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/internal/arangodb/edge"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Read(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, resp, err := edge.Read(ctx, s.client, request.GetCollection(), request.GetKey())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
response.Payload, err = json.Marshal(resp)
return
}

View File

@ -0,0 +1,26 @@
// Copyright 2022 Listware
package edge
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb/edge"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Remove(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, err := edge.Remove(ctx, s.client, request.GetCollection(), request.GetKey())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
return
}

View File

@ -0,0 +1,26 @@
// Copyright 2022 Listware
package edge
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/internal/arangodb/edge"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Update(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, resp, err := edge.Update(ctx, s.client, request.GetCollection(), request.GetKey(), request.GetPayload())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
response.Payload, err = json.Marshal(resp)
return
}

View File

@ -0,0 +1,23 @@
// Copyright 2022 Listware
package finder
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbfinder"
driver "github.com/arangodb/go-driver"
)
type Server struct {
pbfinder.UnimplementedFinderServiceServer
client driver.Client
}
func New(ctx context.Context) (s *Server, err error) {
s = &Server{}
s.client, err = arangodb.Connect()
return
}

View File

@ -0,0 +1,58 @@
// Copyright 2022 Listware
package finder
import (
"context"
"encoding/json"
"fmt"
"strings"
"git.fg-tech.ru/listware/cmdb/internal/arangodb/query"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbfinder"
)
func (s *Server) Links(ctx context.Context, request *pbfinder.Request) (response *pbfinder.Response, err error) {
response = &pbfinder.Response{}
vars := make(map[string]any)
var args []string
if request.From != "" {
args = append(args, "t._from == @from")
vars["from"] = request.From
}
if request.To != "" {
args = append(args, "t._to == @to")
vars["to"] = request.To
}
if request.Name != "" {
args = append(args, "t._name == @name")
vars["name"] = request.Name
}
metas, resp, err := query.Query(ctx, s.client, fmt.Sprintf("FOR t IN links FILTER %s RETURN t", strings.Join(args, " && ")), vars)
if err != nil {
return
}
for i, meta := range metas {
r := &pbcmdb.Response{}
r.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
r.Payload, err = json.Marshal(resp[i])
if err != nil {
return
}
response.Links = append(response.Links, r)
}
return
}

378
internal/cmdb/qdsl/aql.go Normal file
View File

@ -0,0 +1,378 @@
// Copyright 2022 Listware
package qdsl
import (
"fmt"
"math"
"strconv"
"strings"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
var (
pathLinks = []string{"links"}
rootName = "root"
)
type refSpec struct {
specialLevels, queryLevel int
hasCatchall bool
}
type refPath struct {
v, e string
}
func omitRoot(qdsl Path) Path {
rootBlock := &Block{Node: &Node{Name: &rootName}}
max := len(qdsl) - 1
if max == -1 {
return append(qdsl, rootBlock)
}
last := qdsl[max]
if last.Node == nil {
return append(qdsl, rootBlock)
}
if last.Node.Name == nil {
return append(qdsl, rootBlock)
}
if *last.Node.Name != rootName {
return append(qdsl, rootBlock)
}
return qdsl
}
func pathToAql(element *Element, options *pbqdsl.Options) {
path := omitRoot(element.Path)
max := len(path) - 1
root := path[max]
qdsl := path[:max]
length := len(qdsl)
hasPath := length > 0
// <...
hasCatchallValue := 0
hasCatchall := hasPath && !!qdsl[0].Catchall
if hasCatchall {
hasCatchallValue = 1
}
specialLevels := getSpecialDepth(options)
queryLevel := int(math.Max(float64(length-hasCatchallValue-specialLevels), 0))
queryLevelMax := queryLevel
qdslBase := make(Path, len(qdsl))
copy(qdslBase, qdsl)
if hasCatchall {
queryLevelMax = int(math.Max(float64(queryLevel), 10))
qdslBase := make(Path, len(qdsl)-1)
copy(qdslBase, qdsl[1:])
}
var aqlFraments []string
// объект, линк, путь на уровень 1..1
// начальный объект
// коллекиця линков и/или граф
aqlFraments = append(aqlFraments, fmt.Sprintf("for object, link, path in %d..%d outbound 'system/%s' graph system\n", queryLevel, queryLevelMax, *root.Node.Name))
// непонятная магия, которая вроде бы не влияет на результат, но добавляет задержку
// aqlFraments = append(aqlFraments, autoRestrict("path", false))
// protect against missing object
aqlFraments = append(aqlFraments, "filter object\n")
reverse(qdslBase)
for i, level := range qdslBase {
res := convertLevelToAQL(level, i, refSpec{specialLevels, queryLevel, hasCatchall})
aqlFraments = append(aqlFraments, res)
}
if hasCatchall {
res := convertLevelToAQL(qdsl[0], -1, refSpec{specialLevels, queryLevel, hasCatchall})
aqlFraments = append(aqlFraments, res)
}
querySearch := assembleFrags(aqlFraments, "\n")
var returnList []string
if options.Object {
returnList = append(returnList, "object: object")
}
if options.Id {
returnList = append(returnList, "id: object._id")
}
if options.Key {
returnList = append(returnList, "key: object._key")
}
if options.Link {
returnList = append(returnList, "link: link")
}
if options.LinkId {
returnList = append(returnList, "link_id: link._id")
}
if options.Name {
returnList = append(returnList, "name: link._name")
}
if options.Type {
returnList = append(returnList, "type: link._type")
}
if options.Path {
returnList = append(returnList, "path: path")
}
returnExpr := assembleFrags(returnList, ",\n ")
element.Query = fmt.Sprintf(`
%s
return {
%s
}
`, querySearch, returnExpr)
}
func getSpecialDepth(options *pbqdsl.Options) int {
return 0
}
func autoRestrict(pathVar string, reverse bool) string {
var reverseStr = "iv"
if reverse {
reverseStr = "0, iv"
}
return fmt.Sprintf(
`
filter (for iv in 0..(length(%s.vertices) - 1)
let r = %s.vertices[iv]._meta.restrict
return iv < 0 || !r || (for e in slice(%s.edges, %s)
return parse_identifier(e).collection) all in r
) all == true`, pathVar, pathVar, pathVar, reverseStr)
}
func assembleFrags(frags []string, separator string) string {
// FIXME remove '\n' from array
return strings.Join(frags, separator)
}
func convertLevelToAQL(block *Block, i int, refSpec refSpec) string {
refs := getRefPath(i, refSpec)
var nameFilter string
if block.Node != nil {
names := enrollNodeName(block.Node)
var condition string
if len(names) == 1 {
condition = fmt.Sprintf("== '%s'", names[0])
} else {
condition = fmt.Sprintf("in [%s]", strings.Join(names, ","))
}
nameFilter = fmt.Sprintf("filter %s._name %s\n", refs.e, condition)
}
var attrFilter []string
if block.Filter != nil {
for _, filter := range block.Filter.Filter {
var items []string
for _, exp := range filter {
prefix := exp.Expression.Variable[0]
propPath := exp.Expression.Variable[1:]
var vertexPivot bool
if prefix == "@" || prefix == "object" {
vertexPivot = true
}
var edgePivot bool
if prefix == "$" || prefix == "link" {
edgePivot = true
}
pivot := prefix
if vertexPivot {
pivot = refs.v
} else if edgePivot {
pivot = refs.e
}
var propAccess string
for _, value := range propPath {
propAccess = fmt.Sprintf("%s.%s", propAccess, value)
}
var result []string
result = append(result, fmt.Sprintf("%s%s", pivot, propAccess))
result = append(result, exp.Expression.Op)
result = append(result, exp.Expression.Evaluation)
result = append(result, exp.BoolOp)
items = append(items, assembleFrags(result, " "))
}
attrFilter = append(attrFilter, "filter "+assembleFrags(items, " "))
}
}
return nameFilter + assembleFrags(attrFilter, "\n")
}
func single(pathVar string, i int) refPath {
j := i
//if i > 0 {
j = i + 1
//}
return refPath{
v: fmt.Sprintf("%s.vertices[%d]", pathVar, j),
e: fmt.Sprintf("%s.edges[%d]", pathVar, i),
}
}
func getRefPath(i int, refSpec refSpec) refPath {
if refSpec.specialLevels == 0 || (i >= 0 && i < refSpec.queryLevel) {
return single("path", i)
}
if !refSpec.hasCatchall && i >= refSpec.queryLevel {
return single("specialPath", i-refSpec.queryLevel)
}
if i < 0 {
if i < -refSpec.specialLevels {
return single("path", i+refSpec.specialLevels)
} else {
return single("specialPath", i)
}
}
// includeDescendants && specialLevels > 0 && i >= queryLevel
return refPath{
v: fmt.Sprintf("(path.vertices[%d] || specialPath.vertices[%d - length(path.vertices)])", i+1, i+1),
e: fmt.Sprintf("(path.edges[%d] || specialPath.edges[%d - length(path.edges)])", i, i),
}
}
func enrollNodeName(node *Node) (names []string) {
if node.Ranges == nil {
names = append(names, *node.Name)
return
}
for _, ranges := range node.Ranges {
from := *ranges.From
if ranges.To == nil {
if node.Name == nil {
names = append(names, from)
} else {
names = append(names, *node.Name+from)
}
} else {
to := *ranges.To
var isPadded bool
var fixedSize int
if from[0] == '0' {
isPadded = true
fixedSize = len(from)
}
fromInt, _ := strconv.Atoi(from)
toInt, _ := strconv.Atoi(to)
for i := fromInt; i <= toInt; i++ {
if isPadded {
names = append(names, fmt.Sprintf("\"%s%s\"", *node.Name, padZeroes(i, fixedSize)))
} else {
names = append(names, fmt.Sprintf("\"%s%d\"", *node.Name, i))
}
}
}
}
return
}
func padZeroes(i, paddedSize int) (s string) {
s = fmt.Sprint(i)
for len(s) < paddedSize {
s = "0" + s
}
return
}
func getQuerySort(qdsl *Block) (s string) {
if qdsl == nil {
return
}
return
}
/*
function getQueryLimit(qdsl) {
const l = (qdsl && qdsl.limits) ? qdsl.limits : null;
return l
? `limit ${l.offset}, ${l.limit}`
: '';
}
function getQuerySort(qdsl) {
const sortStr = (!qdsl || !qdsl.sort || qdsl.sort.length === 0)
? null
: qdsl.sort.reduce((str, el) => {
let currentPrefix;
const [prefix, ...rest] = el.field;
switch(prefix) {
case '@':
case 'object':
currentPrefix = 'rObject';
break;
case '$':
case 'link':
currentPrefix = 'rLink';
break;
case 'path':
currentPrefix = 'rPath';
break;
default:
currentPrefix = prefix;
}
const fieldStr = [currentPrefix, ...rest].join('.');
return str
? `${str}, ${fieldStr} ${el.direction}`
: `sort ${fieldStr} ${el.direction}`;
}, null);
return sortStr ? sortStr + '\n' : '';
}*/

View File

@ -0,0 +1,27 @@
// Copyright 2022 Listware
package qdsl
import (
"testing"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
func TestPathToAql(t *testing.T) {
query := &pbqdsl.Query{
Query: "*[?$._name == 'init'?].exmt.functions.objects",
Options: &pbqdsl.Options{
LinkId: true,
Object: true,
},
}
elements, err := parse(query)
if err != nil {
t.Fatal(err)
}
for _, element := range elements {
t.Log(element.Query)
}
}

View File

@ -0,0 +1,11 @@
// Copyright 2022 Listware
package lexer
import (
"github.com/bbuck/go-lexer"
)
type QdslLexer struct {
lexer.L
}

View File

@ -0,0 +1,5 @@
// Copyright 2022 Listware
package qdsl
type Link map[string]any

View File

@ -0,0 +1,5 @@
// Copyright 2022 Listware
package qdsl
type Object map[string]any

View File

@ -0,0 +1,62 @@
// Copyright 2022 Listware
package qdsl
import (
"context"
"sort"
"strings"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/documents"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
func parse(query *pbqdsl.Query) (elements []*Element, err error) {
got, err := ParseReader("", strings.NewReader(query.GetQuery()))
if err != nil {
return
}
if query.GetOptions() == nil {
query.Options = &pbqdsl.Options{}
}
normalizeOptions(query.GetOptions())
for _, i := range got.([]any) {
if element, ok := i.(*Element); ok {
pathToAql(element, query.GetOptions())
elements = append(elements, element)
}
}
sort.Slice(elements, func(i, j int) bool {
return elements[i].Action > elements[j].Action
})
return
}
func (s *Server) query(ctx context.Context, element *Element) (nodes documents.Nodes, err error) {
db, err := arangodb.Database(ctx, s.client)
if err != nil {
return
}
cursor, err := db.Query(ctx, element.Query, nil)
if err != nil {
return
}
defer cursor.Close()
for cursor.HasMore() {
var node documents.Node
if _, err = cursor.ReadDocument(ctx, &node); err != nil {
return
}
nodes.Add(&node)
}
return
}

View File

@ -0,0 +1,24 @@
// Copyright 2022 Listware
package qdsl
import (
"testing"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
func TestQdslToAql(t *testing.T) {
query := &pbqdsl.Query{
Query: "*[?$._from == '47e98408-3d47-4730-ba94-c2314ce1982e'?]",
Options: &pbqdsl.Options{},
}
elements, err := parse(query)
if err != nil {
t.Fatal(err)
}
for _, element := range elements {
t.Log(element.Query)
}
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,114 @@
// Copyright 2022 Listware
package qdsl
import (
"context"
"crypto/tls"
"net/http"
"os"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
driver "github.com/arangodb/go-driver"
arangohttp "github.com/arangodb/go-driver/http"
)
var (
arangoAddr string
arangoUser string
arangoPassword string
)
func init() {
if value, ok := os.LookupEnv("ARANGO_ADDR"); ok {
arangoAddr = value
}
if value, ok := os.LookupEnv("ARANGO_USER"); ok {
arangoUser = value
}
if value, ok := os.LookupEnv("ARANGO_PASSWORD"); ok {
arangoPassword = value
}
}
type Server struct {
pbqdsl.UnimplementedQdslServiceServer
client driver.Client
}
func New(ctx context.Context) (s *Server, err error) {
s = &Server{}
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
// Open a client connection
conn, err := arangohttp.NewConnection(arangohttp.ConnectionConfig{
Transport: tr,
Endpoints: []string{arangoAddr},
})
if err != nil {
return
}
s.client, err = driver.NewClient(driver.ClientConfig{
Connection: conn,
Authentication: driver.BasicAuthentication(arangoUser, arangoPassword),
})
return
}
func (s *Server) Qdsl(ctx context.Context, query *pbqdsl.Query) (elements *pbqdsl.Elements, err error) {
elements = &pbqdsl.Elements{Elements: make([]*pbqdsl.Element, 0)}
qdslElements, err := parse(query)
if err != nil {
return
}
graph, err := arangodb.Graph(ctx, s.client)
if err != nil {
return
}
for _, element := range qdslElements {
documents, err := s.query(ctx, element)
if err != nil {
return elements, err
}
for _, document := range documents {
elements.Elements = append(elements.Elements, document.ToElement())
if query.Options.Remove {
if query.Options.Id {
collection, err := graph.VertexCollection(ctx, document.Id.Collection())
if err != nil {
return elements, err
}
if _, err := collection.RemoveDocument(ctx, document.Id.Key()); err != nil {
return elements, err
}
}
if query.Options.LinkId {
collection, _, err := graph.EdgeCollection(ctx, document.LinkId.Collection())
if err != nil {
return elements, err
}
if _, err := collection.RemoveDocument(ctx, document.LinkId.Key()); err != nil {
return elements, err
}
}
}
}
}
return
}

View File

@ -0,0 +1,25 @@
// Copyright 2022 Listware
package qdsl
import (
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
func normalizeOptions(options *pbqdsl.Options) {
if !options.GetId() &&
!options.GetKey() &&
!options.GetName() &&
!options.GetType() &&
!options.GetLink() &&
!options.GetLinkId() &&
!options.GetPath() &&
!options.GetObject() {
options.Id = true
}
}
func reverse[S ~[]E, E any](s S) {
for i, j := 0, len(s)-1; i < j; i, j = i+1, j-1 {
s[i], s[j] = s[j], s[i]
}
}

View File

@ -0,0 +1,23 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
driver "github.com/arangodb/go-driver"
)
type Server struct {
pbcmdb.UnimplementedVertexServiceServer
client driver.Client
}
func New(ctx context.Context) (s *Server, err error) {
s = &Server{}
s.client, err = arangodb.Connect()
return
}

View File

@ -0,0 +1,26 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"encoding/json"
vertex "git.fg-tech.ru/listware/cmdb/internal/arangodb/vertex"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Create(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, resp, err := vertex.Create(ctx, s.client, request.GetCollection(), request.GetPayload())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
response.Payload, err = json.Marshal(resp)
return
}

View File

@ -0,0 +1,26 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"encoding/json"
vertex "git.fg-tech.ru/listware/cmdb/internal/arangodb/vertex"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Read(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, resp, err := vertex.Read(ctx, s.client, request.GetCollection(), request.GetKey())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
response.Payload, err = json.Marshal(resp)
return
}

View File

@ -0,0 +1,24 @@
// Copyright 2022 Listware
package vertex
import (
"context"
vertex "git.fg-tech.ru/listware/cmdb/internal/arangodb/vertex"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Remove(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, err := vertex.Remove(ctx, s.client, request.GetCollection(), request.GetKey())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
return
}

View File

@ -0,0 +1,26 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"encoding/json"
vertex "git.fg-tech.ru/listware/cmdb/internal/arangodb/vertex"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func (s *Server) Update(ctx context.Context, request *pbcmdb.Request) (response *pbcmdb.Response, err error) {
response = &pbcmdb.Response{}
meta, resp, err := vertex.Update(ctx, s.client, request.GetCollection(), request.GetKey(), request.GetPayload())
if err != nil {
return
}
response.Meta = &pbcmdb.Meta{
Key: meta.Key,
Id: meta.ID.String(),
Rev: meta.Rev,
}
response.Payload, err = json.Marshal(resp)
return
}

513
internal/schema/reflect.go Normal file
View File

@ -0,0 +1,513 @@
package schema
import (
"encoding/json"
"net"
"net/url"
"reflect"
"regexp"
"strconv"
"strings"
"time"
)
// Version is the JSON Schema version.
// If extending JSON Schema with custom values use a custom URI.
// RFC draft-wright-json-schema-00, section 6
var Version = "http://json-schema.org/draft-04/schema#"
// Schema is the root schema.
// RFC draft-wright-json-schema-00, section 4.5
type Schema struct {
*Type
Definitions Definitions `json:"definitions,omitempty"`
}
// Type represents a JSON Schema object type.
type Type struct {
// RFC draft-wright-json-schema-00
// Version string `json:"$schema,omitempty"` // section 6.1
Version string `json:"-"` // section 6.1
Ref string `json:"$ref,omitempty"` // section 7
// RFC draft-wright-json-schema-validation-00, section 5
MultipleOf int `json:"multipleOf,omitempty"` // section 5.1
Maximum int `json:"maximum,omitempty"` // section 5.2
ExclusiveMaximum bool `json:"exclusiveMaximum,omitempty"` // section 5.3
Minimum int `json:"minimum,omitempty"` // section 5.4
ExclusiveMinimum bool `json:"exclusiveMinimum,omitempty"` // section 5.5
MaxLength int `json:"maxLength,omitempty"` // section 5.6
MinLength int `json:"minLength,omitempty"` // section 5.7
Pattern string `json:"pattern,omitempty"` // section 5.8
AdditionalItems *Type `json:"additionalItems,omitempty"` // section 5.9
Items *Type `json:"items,omitempty"` // section 5.9
MaxItems int `json:"maxItems,omitempty"` // section 5.10
MinItems int `json:"minItems,omitempty"` // section 5.11
UniqueItems bool `json:"uniqueItems,omitempty"` // section 5.12
MaxProperties int `json:"maxProperties,omitempty"` // section 5.13
MinProperties int `json:"minProperties,omitempty"` // section 5.14
Required []string `json:"required,omitempty"` // section 5.15
Properties map[string]*Type `json:"properties,omitempty"` // section 5.16
PatternProperties map[string]*Type `json:"patternProperties,omitempty"` // section 5.17
AdditionalProperties json.RawMessage `json:"additionalProperties,omitempty"` // section 5.18
Dependencies map[string]*Type `json:"dependencies,omitempty"` // section 5.19
Enum []interface{} `json:"enum,omitempty"` // section 5.20
Type string `json:"type,omitempty"` // section 5.21
AllOf []*Type `json:"allOf,omitempty"` // section 5.22
AnyOf []*Type `json:"anyOf,omitempty"` // section 5.23
OneOf []*Type `json:"oneOf,omitempty"` // section 5.24
Not *Type `json:"not,omitempty"` // section 5.25
Definitions Definitions `json:"definitions,omitempty"` // section 5.26
// RFC draft-wright-json-schema-validation-00, section 6, 7
Title string `json:"title,omitempty"` // section 6.1
Description string `json:"description,omitempty"` // section 6.1
Default interface{} `json:"default,omitempty"` // section 6.2
Format string `json:"format,omitempty"` // section 7
// RFC draft-wright-json-schema-hyperschema-00, section 4
Media *Type `json:"media,omitempty"` // section 4.3
BinaryEncoding string `json:"binaryEncoding,omitempty"` // section 4.3
}
// Reflect reflects to Schema from a value using the default Reflector
func Reflect(v interface{}) *Schema {
return ReflectFromType(reflect.TypeOf(v))
}
// ReflectFromType generates root schema using the default Reflector
func ReflectFromType(t reflect.Type) *Schema {
r := &Reflector{}
return r.ReflectFromType(t)
}
// A Reflector reflects values into a Schema.
type Reflector struct {
// AllowAdditionalProperties will cause the Reflector to generate a schema
// with additionalProperties to 'true' for all struct types. This means
// the presence of additional keys in JSON objects will not cause validation
// to fail. Note said additional keys will simply be dropped when the
// validated JSON is unmarshaled.
AllowAdditionalProperties bool
// RequiredFromJSONSchemaTags will cause the Reflector to generate a schema
// that requires any key tagged with `jsonschema:required`, overriding the
// default of requiring any key *not* tagged with `json:,omitempty`.
RequiredFromJSONSchemaTags bool
// ExpandedStruct will cause the toplevel definitions of the schema not
// be referenced itself to a definition.
ExpandedStruct bool
// TitleNotation will cause the Reflector to generate a schema
// with different title notations. This means
// the title will be transform your structure name from
// camelcase(go-case) notation to the declared. There are
// two supported notations: snake and dash.
// P.S. first letter capitalisation will be ignored (converted to lower-case)
TitleNotation string
}
// Reflect reflects to Schema from a value.
func (r *Reflector) Reflect(v interface{}) *Schema {
return r.ReflectFromType(reflect.TypeOf(v))
}
// ReflectTitle return only structure name as profile type title
func (r *Reflector) ReflectTitle(v interface{}) string {
return r.getTitle(reflect.TypeOf(v))
}
func (r *Reflector) getTitle(t reflect.Type) string {
name := t.Name()
if t.Kind() == reflect.Ptr {
name = t.Elem().Name()
}
var matchAllCap = regexp.MustCompile("([a-z0-9])([A-Z])")
switch r.TitleNotation {
case "dash":
return strings.ToLower(matchAllCap.ReplaceAllString(name, "${1}-${2}"))
case "snake":
return strings.ToLower(matchAllCap.ReplaceAllString(name, "${1}_${2}"))
default:
return name
}
}
// ReflectFromType generates root schema
func (r *Reflector) ReflectFromType(t reflect.Type) *Schema {
definitions := Definitions{}
if r.ExpandedStruct {
st := &Type{
Version: Version,
Title: r.getTitle(t),
Type: "object",
Properties: map[string]*Type{},
AdditionalProperties: []byte("false"),
}
if r.AllowAdditionalProperties {
st.AdditionalProperties = []byte("true")
}
r.reflectStructFields(st, definitions, t)
r.reflectStruct(definitions, t)
delete(definitions, t.Name())
return &Schema{Type: st, Definitions: definitions}
}
s := &Schema{
Type: r.reflectTypeToSchema(definitions, t),
Definitions: definitions,
}
return s
}
// Definitions hold schema definitions.
// http://json-schema.org/latest/json-schema-validation.html#rfc.section.5.26
// RFC draft-wright-json-schema-validation-00, section 5.26
type Definitions map[string]*Type
// Available Go defined types for JSON Schema Validation.
// RFC draft-wright-json-schema-validation-00, section 7.3
var (
timeType = reflect.TypeOf(time.Time{}) // date-time RFC section 7.3.1
ipType = reflect.TypeOf(net.IP{}) // ipv4 and ipv6 RFC section 7.3.4, 7.3.5
uriType = reflect.TypeOf(url.URL{}) // uri RFC section 7.3.6
)
// Byte slices will be encoded as base64
var byteSliceType = reflect.TypeOf([]byte(nil))
// Go code generated from protobuf enum types should fulfil this interface.
type protoEnum interface {
EnumDescriptor() ([]byte, []int)
}
var protoEnumType = reflect.TypeOf((*protoEnum)(nil)).Elem()
func (r *Reflector) reflectTypeToSchema(definitions Definitions, t reflect.Type) *Type {
// Already added to definitions?
if _, ok := definitions[t.Name()]; ok {
return &Type{Ref: "#/definitions/" + t.Name()}
}
// jsonpb will marshal protobuf enum options as either strings or integers.
// It will unmarshal either.
if t.Implements(protoEnumType) {
return &Type{OneOf: []*Type{
{Type: "string"},
{Type: "integer"},
}}
}
// Defined format types for JSON Schema Validation
// RFC draft-wright-json-schema-validation-00, section 7.3
// TODO email RFC section 7.3.2, hostname RFC section 7.3.3, uriref RFC section 7.3.7
switch t {
case ipType:
// TODO differentiate ipv4 and ipv6 RFC section 7.3.4, 7.3.5
return &Type{Type: "string", Format: "ipv4"} // ipv4 RFC section 7.3.4
}
switch t.Kind() {
case reflect.Struct:
switch t {
case timeType: // date-time RFC section 7.3.1
return &Type{Type: "string", Format: "date-time"}
case uriType: // uri RFC section 7.3.6
return &Type{Type: "string", Format: "uri"}
default:
return r.reflectStruct(definitions, t)
}
case reflect.Map:
rt := &Type{
Type: "object",
PatternProperties: map[string]*Type{
".*": r.reflectTypeToSchema(definitions, t.Elem()),
},
}
delete(rt.PatternProperties, "additionalProperties")
return rt
case reflect.Slice, reflect.Array:
returnType := &Type{}
if t.Kind() == reflect.Array {
returnType.MinItems = t.Len()
returnType.MaxItems = returnType.MinItems
}
switch t {
case byteSliceType:
returnType.Type = "string"
returnType.Media = &Type{BinaryEncoding: "base64"}
return returnType
default:
returnType.Type = "array"
returnType.Items = r.reflectTypeToSchema(definitions, t.Elem())
return returnType
}
case reflect.Interface:
return &Type{
Type: "object",
AdditionalProperties: []byte("true"),
}
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return &Type{Type: "integer"}
case reflect.Float32, reflect.Float64:
return &Type{Type: "number"}
case reflect.Bool:
return &Type{Type: "boolean"}
case reflect.String:
return &Type{Type: "string"}
case reflect.Ptr:
return r.reflectTypeToSchema(definitions, t.Elem())
}
panic("unsupported type " + t.String())
}
// Refects a struct to a JSON Schema type.
func (r *Reflector) reflectStruct(definitions Definitions, t reflect.Type) *Type {
st := &Type{
Type: "object",
Properties: map[string]*Type{},
AdditionalProperties: []byte("false"),
}
if r.AllowAdditionalProperties {
st.AdditionalProperties = []byte("true")
}
definitions[t.Name()] = st
r.reflectStructFields(st, definitions, t)
return &Type{
Version: Version,
Ref: "#/definitions/" + t.Name(),
}
}
func (r *Reflector) reflectStructFields(st *Type, definitions Definitions, t reflect.Type) {
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
// Skip non Struct types
// https://cs.opensource.google/go/go/+/refs/tags/go1.18:src/reflect/value.go;l=1219
if t.Kind() != reflect.Struct {
return
}
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if !f.IsExported() {
continue
}
// anonymous and exported type should be processed recursively
// current type should inherit properties of anonymous one
if f.Anonymous && (f.Type.Kind() == reflect.Struct || f.Type.Kind() == reflect.Ptr) {
r.reflectStructFields(st, definitions, f.Type)
continue
}
name, required := r.reflectFieldName(f)
if name == "" {
continue
}
property := r.reflectTypeToSchema(definitions, f.Type)
property.structKeywordsFromTags(f)
property.defaultValueFromTags(f)
st.Properties[name] = property
if required {
st.Required = append(st.Required, name)
}
}
}
func (t *Type) defaultValueFromTags(f reflect.StructField) {
def := f.Tag.Get("default") // Get default value
switch t.Type {
case "bool", "boolean":
if def == "true" {
t.Default = true
} else if def == "false" {
t.Default = false
}
case "string":
if def != "" {
t.Default = def
}
case "number", "integer":
if num, err := strconv.Atoi(def); err == nil {
t.Default = num
}
case "array":
// TODO: implement default values for arrays
}
}
func (t *Type) structKeywordsFromTags(f reflect.StructField) {
tags := strings.Split(f.Tag.Get("jsonschema"), ",")
switch t.Type {
case "string":
t.stringKeywords(tags)
case "number", "integer":
t.numbericKeywords(tags)
case "array":
t.arrayKeywords(tags)
}
}
// read struct tags for string type keyworks
func (t *Type) stringKeywords(tags []string) {
for _, tag := range tags {
nameValue := strings.Split(tag, "=")
if len(nameValue) == 2 {
name, val := nameValue[0], nameValue[1]
switch name {
case "minLength":
i, _ := strconv.Atoi(val)
t.MinLength = i
case "maxLength":
i, _ := strconv.Atoi(val)
t.MaxLength = i
case "format":
switch val {
case "date-time", "email", "hostname", "ipv4", "ipv6", "uri":
t.Format = val
break
}
}
}
}
}
// read struct tags for numberic type keyworks
func (t *Type) numbericKeywords(tags []string) {
for _, tag := range tags {
nameValue := strings.Split(tag, "=")
if len(nameValue) == 2 {
name, val := nameValue[0], nameValue[1]
switch name {
case "multipleOf":
i, _ := strconv.Atoi(val)
t.MultipleOf = i
case "minimum":
i, _ := strconv.Atoi(val)
t.Minimum = i
case "maximum":
i, _ := strconv.Atoi(val)
t.Maximum = i
case "exclusiveMaximum":
b, _ := strconv.ParseBool(val)
t.ExclusiveMaximum = b
case "exclusiveMinimum":
b, _ := strconv.ParseBool(val)
t.ExclusiveMinimum = b
}
}
}
}
// read struct tags for object type keyworks
// func (t *Type) objectKeywords(tags []string) {
// for _, tag := range tags{
// nameValue := strings.Split(tag, "=")
// name, val := nameValue[0], nameValue[1]
// switch name{
// case "dependencies":
// t.Dependencies = val
// break;
// case "patternProperties":
// t.PatternProperties = val
// break;
// }
// }
// }
// read struct tags for array type keyworks
func (t *Type) arrayKeywords(tags []string) {
for _, tag := range tags {
nameValue := strings.Split(tag, "=")
if len(nameValue) == 2 {
name, val := nameValue[0], nameValue[1]
switch name {
case "minItems":
i, _ := strconv.Atoi(val)
t.MinItems = i
case "maxItems":
i, _ := strconv.Atoi(val)
t.MaxItems = i
case "uniqueItems":
t.UniqueItems = true
}
}
}
}
func requiredFromJSONTags(tags []string) bool {
if ignoredByJSONTags(tags) {
return false
}
for _, tag := range tags[1:] {
if tag == "omitempty" {
return false
}
}
return true
}
func requiredFromJSONSchemaTags(tags []string) bool {
if ignoredByJSONSchemaTags(tags) {
return false
}
for _, tag := range tags {
if tag == "required" {
return true
}
}
return false
}
func ignoredByJSONTags(tags []string) bool {
return tags[0] == "-"
}
func ignoredByJSONSchemaTags(tags []string) bool {
return tags[0] == "-"
}
func (r *Reflector) reflectFieldName(f reflect.StructField) (string, bool) {
if !f.IsExported() { // unexported field, ignore it
return "", false
}
jsonTags := strings.Split(f.Tag.Get("json"), ",")
if ignoredByJSONTags(jsonTags) {
return "", false
}
jsonSchemaTags := strings.Split(f.Tag.Get("jsonschema"), ",")
if ignoredByJSONSchemaTags(jsonSchemaTags) {
return "", false
}
name := f.Name
required := requiredFromJSONTags(jsonTags)
if r.RequiredFromJSONSchemaTags {
required = requiredFromJSONSchemaTags(jsonSchemaTags)
}
if jsonTags[0] != "" {
name = jsonTags[0]
}
return name, required
}

130
internal/server/server.go Normal file
View File

@ -0,0 +1,130 @@
// Copyright 2022 Listware
package server
import (
"context"
"fmt"
"net"
"os"
"os/signal"
"syscall"
"git.fg-tech.ru/listware/cmdb/internal/arangodb"
"git.fg-tech.ru/listware/cmdb/internal/cmdb/edge"
"git.fg-tech.ru/listware/cmdb/internal/cmdb/finder"
"git.fg-tech.ru/listware/cmdb/internal/cmdb/qdsl"
"git.fg-tech.ru/listware/cmdb/internal/cmdb/vertex"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbfinder"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
"google.golang.org/grpc"
)
// set max 100 MB
const maxMsgSize = 100 * 1024 * 1024
var (
cmdbAddr = "127.0.0.1"
cmdbPort = "31415"
)
func init() {
if value, ok := os.LookupEnv("CMDB_ADDR"); ok {
cmdbAddr = value
}
if value, ok := os.LookupEnv("CMDB_PORT"); ok {
cmdbPort = value
}
}
func New() {
ctx, cancel := context.WithCancel(context.Background())
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan,
syscall.SIGINT,
syscall.SIGTERM,
syscall.SIGQUIT,
syscall.SIGHUP,
syscall.SIGUSR1,
syscall.SIGUSR2,
)
go func() {
for {
select {
case sig := <-sigChan:
switch sig {
case syscall.SIGTERM, syscall.SIGQUIT, syscall.SIGINT:
cancel()
}
case <-ctx.Done():
return
}
}
}()
if err := serve(ctx); err != nil {
fmt.Println(err)
return
}
return
}
func serve(ctx context.Context) (err error) {
if err = arangodb.Bootstrap(ctx); err != nil {
return
}
pc, err := net.Listen("tcp", fmt.Sprintf(":%s", cmdbPort))
if err != nil {
return
}
defer pc.Close()
server := grpc.NewServer(
grpc.MaxMsgSize(maxMsgSize),
grpc.MaxRecvMsgSize(maxMsgSize),
grpc.MaxSendMsgSize(maxMsgSize),
)
defer server.Stop()
qdsl, err := qdsl.New(ctx)
if err != nil {
return
}
pbqdsl.RegisterQdslServiceServer(server, pbqdsl.QdslServiceServer(qdsl))
finder, err := finder.New(ctx)
if err != nil {
return
}
pbfinder.RegisterFinderServiceServer(server, pbfinder.FinderServiceServer(finder))
edge, err := edge.New(ctx)
if err != nil {
return
}
pbcmdb.RegisterEdgeServiceServer(server, pbcmdb.EdgeServiceServer(edge))
vertex, err := vertex.New(ctx)
if err != nil {
return
}
pbcmdb.RegisterVertexServiceServer(server, pbcmdb.VertexServiceServer(vertex))
go server.Serve(pc)
<-ctx.Done()
return
}
func Client() (conn *grpc.ClientConn, err error) {
return grpc.Dial(fmt.Sprintf("%s:%s", cmdbAddr, cmdbPort), grpc.WithInsecure(), grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(maxMsgSize), grpc.MaxCallSendMsgSize(maxMsgSize)))
}

View File

@ -0,0 +1,22 @@
// Copyright 2022 Listware
package documents
import (
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
// BaseDocument is a minimal document for use in non-edge collection.
type BaseDocument struct {
Key string `json:"_key,omitempty"`
ID DocumentID `json:"_id,omitempty"`
Rev string `json:"_rev,omitempty"`
}
func NewBaseDocument(meta *pbcmdb.Meta) *BaseDocument {
return &BaseDocument{
Key: meta.GetKey(),
ID: DocumentID(meta.GetId()),
Rev: meta.GetRev(),
}
}

View File

@ -0,0 +1,11 @@
// Copyright 2022 Listware
package documents
// EdgeDocument is a minimal document for use in edge collection.
type EdgeDocument struct {
BaseDocument
From DocumentID `json:"_from,omitempty"`
To DocumentID `json:"_to,omitempty"`
Type string `json:"_type,omitempty"`
}

View File

@ -0,0 +1,81 @@
// Copyright 2022 Listware
package documents
import (
"fmt"
"net/url"
"path"
"strings"
)
// DocumentID is a document ID
// Consists of two parts - collection name and key
type DocumentID string
// Topic representation of DocumentID
func (id DocumentID) Topic() string {
return strings.Replace(id.String(), "/", ".", -1)
}
// Parent - return parent's Document ID
func (id DocumentID) Parent() DocumentID {
return DocumentID(path.Dir(id.String()))
}
// Validate validates the given id
func (id DocumentID) Validate() error {
if id == "" {
return fmt.Errorf("DocumentID is empty")
}
parts := strings.Split(string(id), "/")
if len(parts) < 2 {
return fmt.Errorf("Expected 'collection/key[/profile_name]', got '%s'", string(id))
}
if parts[0] == "" {
return fmt.Errorf("Collection part of '%s' is empty", string(id))
}
if parts[1] == "" {
return fmt.Errorf("Key part of '%s' is empty", string(id))
}
return nil
}
// pathEscape escapes the given value for use in a URL path.
func pathEscape(s string) string {
return url.QueryEscape(s)
}
// pathUnescape unescapes the given value for use in a URL path.
func pathUnescape(s string) string {
r, _ := url.QueryUnescape(s)
return r
}
// Collection returns the collection part of the ID.
func (id DocumentID) Collection() string {
parts := strings.Split(string(id), "/")
return pathUnescape(parts[0])
}
// Key returns the key part of the ID.
func (id DocumentID) Key() string {
parts := strings.Split(string(id), "/")
if len(parts) >= 2 {
return pathUnescape(parts[1])
}
return ""
}
// ProfileName returns the profile name part of the ID.
func (id DocumentID) ProfileName() string {
parts := strings.Split(string(id), "/")
if len(parts) >= 3 {
return pathUnescape(parts[2])
}
return ""
}
func (id DocumentID) String() string {
return string(id)
}

View File

@ -0,0 +1,59 @@
// Copyright 2022 Listware
package documents
import (
"encoding/json"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
type Node struct {
Id DocumentID `json:"id,omitempty"`
LinkId DocumentID `json:"link_id,omitempty"`
Key string `json:"key,omitempty"`
Name string `json:"name,omitempty"`
Type string `json:"type,omitempty"`
Object json.RawMessage `json:"object,omitempty"`
Link json.RawMessage `json:"link,omitempty"`
Path json.RawMessage `json:"path,omitempty"`
}
type Nodes []*Node
func NewNodes() Nodes {
return make([]*Node, 0)
}
func (nodes *Nodes) AddElement(element *pbqdsl.Element) {
*nodes = append(*nodes, NewNode(element))
}
func (nodes *Nodes) Add(node ...*Node) {
*nodes = append(*nodes, node...)
}
func NewNode(element *pbqdsl.Element) *Node {
return &Node{
Id: DocumentID(element.Id),
Key: element.Key,
Name: element.Name,
Type: element.Type,
Object: element.Object,
LinkId: DocumentID(element.LinkId),
Link: element.Link,
Path: element.Path,
}
}
func (node *Node) ToElement() *pbqdsl.Element {
return &pbqdsl.Element{
Id: node.Id.String(),
Key: node.Key,
Name: node.Name,
Type: node.Type,
Object: node.Object,
Link: node.Link,
LinkId: node.LinkId.String(),
Path: node.Path,
}
}

63
pkg/cmdb/edge/client.go Normal file
View File

@ -0,0 +1,63 @@
// Copyright 2022 Listware
package edge
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/server"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
func Create(ctx context.Context, collection string, payload any) (resp *pbcmdb.Response, err error) {
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewEdgeServiceClient(conn)
request := &pbcmdb.Request{Collection: collection}
return client.Create(ctx, request)
}
func Read(ctx context.Context, key, collection string) (resp *pbcmdb.Response, err error) {
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewEdgeServiceClient(conn)
return client.Read(ctx, &pbcmdb.Request{Key: key, Collection: collection})
}
func Update(ctx context.Context, key, collection string, payload any) (resp *pbcmdb.Response, err error) {
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewEdgeServiceClient(conn)
request := &pbcmdb.Request{Key: key, Collection: collection}
return client.Update(ctx, request)
}
func Remove(ctx context.Context, key, collection string) (resp *pbcmdb.Response, err error) {
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewEdgeServiceClient(conn)
request := &pbcmdb.Request{Key: key, Collection: collection}
return client.Remove(ctx, request)
}

View File

@ -0,0 +1,51 @@
// Copyright 2022 Listware
package links
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/documents"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/edge"
)
const (
collection = "links"
)
func Create(ctx context.Context, payload any) (meta *documents.BaseDocument, err error) {
response, err := edge.Create(ctx, collection, payload)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}
func Read(ctx context.Context, key string) (payload json.RawMessage, err error) {
response, err := edge.Read(ctx, key, collection)
if err != nil {
return
}
payload = response.GetPayload()
return
}
func Update(ctx context.Context, key string, payload any) (meta *documents.BaseDocument, err error) {
response, err := edge.Update(ctx, key, collection, payload)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}
func Remove(ctx context.Context, key string) (meta *documents.BaseDocument, err error) {
response, err := edge.Remove(ctx, key, collection)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}

38
pkg/cmdb/qdsl/client.go Normal file
View File

@ -0,0 +1,38 @@
// Copyright 2022 Listware
package qdsl
import (
"context"
"git.fg-tech.ru/listware/cmdb/internal/server"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/documents"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
// RawQdsl query with options as object
func RawQdsl(ctx context.Context, query string, options *pbqdsl.Options) (nodes documents.Nodes, err error) {
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbqdsl.NewQdslServiceClient(conn)
elements, err := client.Qdsl(ctx, &pbqdsl.Query{Query: query, Options: options})
if err != nil {
return
}
nodes = documents.NewNodes()
for _, element := range elements.GetElements() {
nodes.AddElement(element)
}
return
}
// Qdsl query with options OptionsOption array
func Qdsl(ctx context.Context, query string, options ...OptionsOption) (documents.Nodes, error) {
return RawQdsl(ctx, query, NewOptions(options...))
}

82
pkg/cmdb/qdsl/options.go Normal file
View File

@ -0,0 +1,82 @@
// Copyright 2022 Listware
package qdsl
import (
"git.fg-tech.ru/listware/proto/sdk/pbcmdb/pbqdsl"
)
// OptionsOption query options
type OptionsOption func(*pbqdsl.Options)
// NewOptions return new query options
func NewOptions(opts ...OptionsOption) *pbqdsl.Options {
h := &pbqdsl.Options{}
for _, opt := range opts {
opt(h)
}
return h
}
// WithId return vertex '_id', default true
func WithId() OptionsOption {
return func(h *pbqdsl.Options) {
h.Id = true
}
}
// WithKey return vertex '_key', default false
func WithKey() OptionsOption {
return func(h *pbqdsl.Options) {
h.Key = true
}
}
// WithName return edge '_name', default false
func WithName() OptionsOption {
return func(h *pbqdsl.Options) {
h.Name = true
}
}
// WithObject return vertex 'object', default false
func WithObject() OptionsOption {
return func(h *pbqdsl.Options) {
h.Object = true
}
}
// WithLink return edge 'object', default false
func WithLink() OptionsOption {
return func(h *pbqdsl.Options) {
h.Link = true
}
}
// WithLinkId return edge '_id', default false
func WithLinkId() OptionsOption {
return func(h *pbqdsl.Options) {
h.LinkId = true
}
}
// WithType return edge '_type', default false
func WithType() OptionsOption {
return func(h *pbqdsl.Options) {
h.Type = true
}
}
// FIXME WithType now disabled
func WithPath() OptionsOption {
return func(h *pbqdsl.Options) {
h.Path = true
}
}
// WithRemove remove all founded results
func WithRemove() OptionsOption {
return func(h *pbqdsl.Options) {
h.Remove = true
}
}

87
pkg/cmdb/vertex/client.go Normal file
View File

@ -0,0 +1,87 @@
// Copyright 2022 Listware
package vertex
import (
"context"
"encoding/json"
"errors"
"git.fg-tech.ru/listware/cmdb/internal/server"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
var (
ErrEmptyPayload = errors.New("empty payload")
)
func Create(ctx context.Context, collection string, payload any) (resp *pbcmdb.Response, err error) {
if payload == nil {
return nil, ErrEmptyPayload
}
payloadRaw, err := json.Marshal(payload)
if err != nil {
return
}
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewVertexServiceClient(conn)
request := &pbcmdb.Request{Collection: collection, Payload: payloadRaw}
return client.Create(ctx, request)
}
func Read(ctx context.Context, key, collection string) (resp *pbcmdb.Response, err error) {
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewVertexServiceClient(conn)
return client.Read(ctx, &pbcmdb.Request{Key: key, Collection: collection})
}
func Update(ctx context.Context, key, collection string, payload any) (resp *pbcmdb.Response, err error) {
if payload == nil {
return nil, ErrEmptyPayload
}
payloadRaw, err := json.Marshal(payload)
if err != nil {
return
}
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewVertexServiceClient(conn)
request := &pbcmdb.Request{Key: key, Collection: collection, Payload: payloadRaw}
return client.Update(ctx, request)
}
func Remove(ctx context.Context, key, collection string) (resp *pbcmdb.Response, err error) {
conn, err := server.Client()
if err != nil {
return
}
defer conn.Close()
client := pbcmdb.NewVertexServiceClient(conn)
request := &pbcmdb.Request{Key: key, Collection: collection}
return client.Remove(ctx, request)
}

View File

@ -0,0 +1,52 @@
// Copyright 2022 Listware
package objects
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/documents"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/vertex"
)
const (
collection = "objects"
)
func Create(ctx context.Context, payload any) (meta *documents.BaseDocument, err error) {
response, err := vertex.Create(ctx, collection, payload)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}
func Read(ctx context.Context, key string) (meta *documents.BaseDocument, payload json.RawMessage, err error) {
response, err := vertex.Read(ctx, key, collection)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
payload = response.GetPayload()
return
}
func Update(ctx context.Context, key string, payload any) (meta *documents.BaseDocument, err error) {
response, err := vertex.Update(ctx, key, collection, payload)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}
func Remove(ctx context.Context, key string) (meta *documents.BaseDocument, err error) {
response, err := vertex.Remove(ctx, key, collection)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}

View File

@ -0,0 +1,52 @@
// Copyright 2022 Listware
package types
import (
"context"
"encoding/json"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/documents"
"git.fg-tech.ru/listware/cmdb/pkg/cmdb/vertex"
)
const (
collection = "types"
)
func Create(ctx context.Context, payload any) (meta *documents.BaseDocument, err error) {
response, err := vertex.Create(ctx, collection, payload)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}
func Read(ctx context.Context, key string) (meta *documents.BaseDocument, payload json.RawMessage, err error) {
response, err := vertex.Read(ctx, key, collection)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
payload = response.GetPayload()
return
}
func Update(ctx context.Context, key string, payload any) (meta *documents.BaseDocument, err error) {
response, err := vertex.Update(ctx, key, collection, payload)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}
func Remove(ctx context.Context, key string) (meta *documents.BaseDocument, err error) {
response, err := vertex.Remove(ctx, key, collection)
if err != nil {
return
}
meta = documents.NewBaseDocument(response.Meta)
return
}

View File

@ -0,0 +1,35 @@
// Copyright 2022 Listware
package types
import (
"git.fg-tech.ru/listware/cmdb/internal/schema"
"git.fg-tech.ru/listware/proto/sdk/pbcmdb"
)
// Type is a object type struct
type Type struct {
Schema *schema.Schema `json:"schema"`
Triggers map[string]map[string]*pbcmdb.Trigger `json:"triggers"`
}
// NewType return new object type
func NewType(schema *schema.Schema) Type {
return Type{
Schema: schema,
Triggers: make(map[string]map[string]*pbcmdb.Trigger),
}
}
// ReflectType get reflected profiletype
func ReflectType(v interface{}) *Type {
r := &schema.Reflector{
AllowAdditionalProperties: true,
ExpandedStruct: true,
TitleNotation: "dash",
}
return &Type{
Schema: r.Reflect(v),
Triggers: make(map[string]map[string]*pbcmdb.Trigger),
}
}

315
qdsl.peg Normal file
View File

@ -0,0 +1,315 @@
{
package qdsl
type Limits struct {
Limit int `json:"limit"`
Offset int `json:"offset"`
}
type Limit struct {
Sort any `json:"sort"`
Limits *Limits `json:"limits"`
}
type Direction struct {
Direction string `json:"direction"`
Field any `json:"field"`
}
type Variable struct {
Variable []string `json:"variable"`
Op string `json:"op"`
Evaluation string `json:"evaluation"`
}
type Expression struct {
Expression *Variable `json:"expression"`
BoolOp string `json:"boolOp"`
}
type Filter struct {
Filter [][]*Expression `json:"filter"`
// ...limits
}
type Range struct {
From *string `json:"from"`
To *string `json:"to"`
}
type Node struct {
Name *string `json:"name"`
Ranges []*Range `json:"ranges"`
}
type Block struct {
*Filter `json:"filter"`
Any bool `json:"any"`
Catchall bool `json:"catchall"`
Node *Node `json:"node"`
IsGroup bool `json:"isGroup"`
Children []*Element `json:"children"`
}
type Path []*Block
type Element struct {
Action string `json:"action"`
Path Path `json:"path"`
RootExpand bool `json:"rootExpand"`
Query string `json:"query"`
}
func toString(i interface{}) string {
if i == nil {
return ""
}
switch i.(type) {
case string:
return i.(string)
default:
return string(i.([]byte))
}
}
func arrayToStringArray(arr interface{}) (result []string) {
for _, i := range arr.([]interface{}) {
result = append(result, toString(i))
}
return
}
}
start = QUERY
QUERY = base:(e:ELEMENT (__ "," __ / [ ]+) { return e, nil })* last:ELEMENT {
return append(base.([]any), last), nil
}
ELEMENT = action:UNARY levels:(l:LEVEL "." { return l, nil })* last:(LEVEL / "_") {
var blocks []*Block
for _, level := range levels.([]any) {
if block, ok := level.(*Block); ok {
blocks = append(blocks, block)
}
}
block, ok := last.(*Block)
// if last == "_" {
if !ok {
return &Element{Action: toString(action), Path: blocks, RootExpand: true}, nil
}
return &Element{Action: toString(action), Path: append(blocks, block)}, nil
}
UNARY = op:("-")? {
if op == "-" {
return "subtract", nil
}
return "add", nil
}
NODE = nodename:NODENAME ranges:NODERANGE? {
name := toString(nodename)
var arr []*Range
if val, ok := ranges.([]interface{}); ok {
for _, a := range val {
arr = append(arr, a.(*Range))
}
}
return &Node{Name: &name, Ranges: arr}, nil
}
/ ranges:NODERANGE {
var arr []*Range
if val, ok := ranges.([]interface{}); ok {
for _, a := range val {
arr = append(arr, a.(*Range))
}
}
return &Node{Name: nil, Ranges: arr}, nil
}
NODENAME = nodename:(head:[a-z0-9] tail:[a-z_0-9\\-]i* {
return toString(head) + strings.Join(arrayToStringArray(tail), ""), nil
} )
NODERANGE = "[" ranges:(from:INT to:("-" to:INT {
return to, nil
})? __ ","? __ {
fromValue := toString(from)
toValue := toString(to)
return &Range{From: &fromValue, To: &toValue}, nil
})+ "]" {
return ranges, nil
}
INT = num:[0-9a-z]i+ {
return strings.Join(arrayToStringArray(num), ""), nil
}
LEVEL = "(" children:QUERY ")" {
var arr []*Element
if val, ok := children.([]interface{}); ok {
for _, a := range val {
arr = append(arr, a.(*Element))
}
}
return &Block{IsGroup: true, Children: arr}, nil
}
/ "<" block:BLOCK {
return &Block{ Any: false, Catchall: true, Filter:block.(*Filter)}, nil
}
/ "*" block:BLOCK {
return &Block{ Any: true, Filter:block.(*Filter)}, nil
}
/ node:NODE block:BLOCK {
return &Block{ Any: false, Node: node.(*Node), Filter:block.(*Filter)}, nil
}
BLOCK = limits:LIMIT filter:SEARCH {
var filters [][]*Expression
for _, i := range filter.([]any) {
var expressions []*Expression
for _, j := range i.([]any) {
expressions = append(expressions, j.(*Expression))
}
filters = append(filters, expressions)
}
return &Filter{filters}, nil}
/ filter:SEARCH limits:LIMIT {
var filters [][]*Expression
for _, i := range filter.([]any) {
var expressions []*Expression
for _, j := range i.([]any) {
expressions = append(expressions, j.(*Expression))
}
filters = append(filters, expressions)
}
return &Filter{filters}, nil}
/ filter:SEARCH {
var filters [][]*Expression
for _, i := range filter.([]any) {
var expressions []*Expression
for _, j := range i.([]any) {
expressions = append(expressions, j.(*Expression))
}
filters = append(filters, expressions)
}
return &Filter{filters}, nil}
LIMIT = "{" sort:SORT limit:NUMBER ".." offset:NUMBER "}" {
return &Limit{ Sort:sort, Limits: &Limits{ limit.(int), offset.(int) }}, nil }
/ "{" sort:SORT limit:NUMBER "}" {
return &Limit{ Sort:sort, Limits: &Limits{ limit.(int) , 0 }}, nil }
/ "{" sort:SORT "}" {
return &Limit{ Sort:sort }, nil }
SORT = (direction:DIRECTION v:VARIABLE ","? __ {
return &Direction{toString(direction) ,v }, nil})*
DIRECTION = d:("^")? {
if d == nil {
return "ASC", nil
}
return "DESC", nil
}
SEARCH = filters:("[?" filter:FILTER "?]"{
return filter, nil
})* {
return filters, nil
}
FILTER = (expression:EXPRESSION __ boolOp:("&&" / "||")? __ {
expressionValue, _ := expression.(*Variable)
return &Expression{expressionValue, toString(boolOp)}, nil
})+
EXPRESSION = variable:VARIABLE __ op:OP __ evaluation:EVALUATION {
var arr []string
for _, i := range variable.([]any) {
arr = append(arr, toString(i))
}
return &Variable{arr, toString(op), toString(evaluation)}, nil
}
OP = "=="
/ "=~"
/ "!~"
/ "<="
/ ">="
/ "<"
/ ">"
/ "!="
/ "IN"i
/ "LIKE"i
/ "NOT LIKE"i
VARIABLE = variable:("object" / "link" / "path" / "@" / "$") attribute:ATTRIBUTE+ {
return append([]any{variable}, attribute.([]any)...), nil
}
ATTRIBUTE = ("." attrname:ATTRNAME {
return attrname, nil })
/ ("[" attrname:(STRING_LITERAL / INT) "]" {
return attrname, nil })
ATTRNAME = attrname:[a-z\\*0-9_\\-]i+ {
var arr []string
for _, a := range attrname.([]any) {
arr = append(arr, toString(a))
}
return strings.Join(arr, ""), nil
}
STRING_LITERAL = "'" text:("\\'"/[^'])+ "'" {
return strings.Join(text.([]string), ""), nil
}
BOOL = "true" / "false" / "True" / "False"
NULL = "null"
EVALUATION = LITERAL
LITERAL = DBL_LITERAL / SNG_LITERAL / NUMBER / BOOL / NULL / ARR
SNG_LITERAL = q1:"'" cc:[^\\']* q2:"'" { return toString(q1) + strings.Join(arrayToStringArray(cc), "") + toString(q2), nil }
DBL_LITERAL = q1:'"' cc:[^\\"]* q2:'"' { return toString(q1) + strings.Join(arrayToStringArray(cc), "") + toString(q2), nil }
NUMBER = num:[0-9]+ tail:('.' [0-9]+)? {
fmt.Println("AAAAAAaa")
arr:= tail.([]string)
var end string
if len(arr) > 0 {
// FIXME [][]arr?
// end = "." + strings.Join(arr[1], "")
}
return strings.Join(num.([]string), "") + end, nil
}
ARR = '[' __
body:(hd:LITERAL items:(__ ',' __ e:LITERAL { return e, nil })* __ {
arr := []any{hd}
return append(arr, items), nil
})?
']' {
bodyArr := body.([]string)
arr := []string{}
if len(bodyArr) > 0 {
arr = append(arr, strings.Join(bodyArr, ","))
}
return arr, nil
}
__ = [ ]*