Initial commit

This commit is contained in:
bytedream 2022-01-09 16:35:39 +01:00
commit 741f63c0af
7 changed files with 557 additions and 0 deletions

5
.example.env Normal file
View File

@ -0,0 +1,5 @@
HOST = 0.0.0.0
PORT = 8080
ENABLE_REGEX = false
MAX_PATTER_LEN = 70

1
.gitignore vendored Normal file
View File

@ -0,0 +1 @@
/target

21
Cargo.toml Normal file
View File

@ -0,0 +1,21 @@
[package]
name = "smartrelease"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
actix-web = { version = "3.3", features = ["rustls"] }
dotenv = "0.15"
env_logger = "0.9"
lazy_static = "1.4"
log = "0.4"
regex = "1.5"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
[profile.release]
lto = true
panic = "abort"
opt-level = "z"

16
Dockerfile Normal file
View File

@ -0,0 +1,16 @@
FROM rust:1.57-alpine
WORKDIR /smartrelease
COPY . .
RUN apk update && \
apk add musl-dev && \
rm -rf /var/cache/apk
RUN cargo build --release && \
ln -s target/release/smartrelease .
EXPOSE 8080
ENTRYPOINT ["./smartrelease"]

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2022 ByteDream
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

204
README.md Normal file
View File

@ -0,0 +1,204 @@
# smartrelease - Generate links to release assets dynamically
[Releases](https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases) are an essential feature of github (or other platforms like [gitea](https://gitea.io)) and the assets you can attach are pretty useful if you want to pre-compile a binary for example.
But linking to these assets directly in your README can be pretty annoying if every of your release asset has a version number in it, for example `program-v1.0.0`, and with every release the version number changes, and you have to change the direct link to it in your README.
And this is where **smartrelease** enters the game.
It provides a simple yet powerful api endpoint which will redirect the user directly to the latest release asset you've specified in the api url.
<p align="center">
<a href="https://github.com/ByteDream/Yamete-Kudasai/releases/latest">
<img src="https://img.shields.io/github/v/release/ByteDream/smartrelease?style=flat-square" alt="Latest release">
</a>
<a href="https://github.com/ByteDream/smartrelease/blob/master/LICENSE">
<img src="https://img.shields.io/github/license/ByteDream/smartrelease?style=flat-square" alt="License">
</a>
<a href="#">
<img src="https://img.shields.io/github/languages/top/ByteDream/smartrelease?style=flat-square" alt="Top language">
</a>
<a href="https://discord.gg/gUWwekeNNg">
<img src="https://img.shields.io/discord/915659846836162561?label=discord&style=flat-square" alt="Discord">
</a>
</p>
## How it's working
<p align="center"><strong>Take me directly to the <a href="#examples">examples</a>.</strong></p>
The api works with a single link which will take the user to the asset of the latest release you've specified.
Its url scheme looks like the following:
<pre>
<code>https://example.com/<a href="#platform">:platform</a>/<a href="#owner">:owner</a>/<a href="#repository">:repository</a>/<a href="#pattern">:pattern</a></code>
</pre>
### platform
The **platform** is the hoster of your git repository.
The current supported platforms are:
- [GitHub](https://github.com)
- [Gitea](https://gitea.com)
Currently, only the official instances are supported by the api (github has no instance for self-hosting anyway) but support for self-hostet instances is planned.
### owner
The **owner** is owner of the repository (mostly you).
### repository
The **repository** is the name of the repository you want to set up the api for.
### pattern
**pattern** is the part where all the magic happens.
The pattern you specify here is used to redirect the user to the asset you want.
To archive this, wildcards as well as regex (see [here](#warnings) for possible dangers with regex) can be used.
---
The following takes mostly care of wildcards since the [official instance](#official-instance) and default configurations are both only supporting wildcards and not regex.
Wildcards are simply nothing other than text fragments / variables / ... (whatever you want to call it) which are getting replaced with their corresponding values from the release name / tag.
These wildcards are currently supported:
- `major`
_The major version number_.
It must always be a number.
In `smartrelease-v1.21.5-rc4`, `1` is the major version number.
**Note**: The first number occurrence will always be matched as `major`.
At the moment I have no idea how I should avoid this and only match the first number if necessary but if you want create a new [issue](https://github.com/ByteDream/smartrelease/issues/new), and I will take further investigations how to solve this.
- `minor`
_The minor version number_.
It must always be a number.
In `smartrelease-v1.21.5-rc4`, `21` is the minor version number.
- `patch`
_The patch version number_.
It must always be a number.
In `smartrelease-v1.21.5-rc4`, `5` is the patch version number.
- `pre`
_The pre-release number_. It can be a mix of numbers and letters (without any special character between).
In `smartrelease-v1.21.5-rc4`, `rc4` is the pre-release number.
- `tag`
_The release tag_. In `https://github.com/ByteDream/smartrelease/releases/tag/v0.1.0`, `v0.1.0` is the tag.
`major`, `minor`, `patch` and `pre` are all version number specific wildcards.
Hence, they are matched descending.
This means `minor` is only matched if `major` is matched, `patch` is only matched if `minor` is matched and so on.
---
I clearly can't name all cases here where the pattern matches your asset name or not, so if you want to check and test which name is support and which not, I suggest you to visit this [website](https://regex101.com/r/gU5vbe/1) and type in your asset name in the big box.
If it gets highlighted somewhere it is supported, if not then not.
In case your asset name is not supported, but you want it to be supported, feel free to create a new [issue](https://github.com/ByteDream/smartrelease/issues/new) or join the [discord server](https://discord.gg/gUWwekeNNg) and submit your asset name, so I can take care and implement it.
## Examples
For the example the [official instance](#official-instance) is used as host.
Latest release for this repo.
The result looks like this: [Latest release](https://smartrelease.bytedream.org/github/ByteDream/smartrelease/smartrelease-v{major}.{minor}.{patch}_linux)
```
[Latest release](https://smartrelease.bytedream.org/github/ByteDream/smartrelease/smartrelease-v{major}.{minor}.{patch}_linux)
```
We can also use [shields.io](https://shields.io) to make it look more appealing to the user.
The result looks like this: [![Latest release](https://img.shields.io/github/v/release/ByteDream/smartrelease?style=flat-square)](https://smartrelease.bytedream.org/github/ByteDream/smartrelease/smartrelease-v{major}.{minor}.{patch}_linux)
```
[![Latest release](https://img.shields.io/github/v/release/ByteDream/smartrelease?style=flat-square)](https://smartrelease.bytedream.org/github/ByteDream/smartrelease/smartrelease-v{major}.{minor}.{patch}_linux)
```
And now with the official Gitea instance (Gitea is a great open-source based alternative to GitHub, if you didn't knew it already)
The result looks like this: [Now with gitea!](https://smartrelease.bytedream.org/gitea/gitea/tea/tea-{major}.{minor}.{patch}-linux-amd64)
```
[Now with gitea!](https://smartrelease.bytedream.org/gitea/gitea/tea/tea-{major}.{minor}.{patch}-linux-amd64)
```
## Hosting
## Official instance
The official instance is hosted on `https://smartrelease.bytedream.org`.
It has regex disabled and a maximal pattern length of 70 character.
So if you want, for example, using the official api for this repo, the following link will do it:
```
https://smartrelease.bytedream.org/github/ByteDream/smartrelease/smartrelease-v{major}.{minor}.{patch}_linux
```
Nevertheless, I recommend you to host your own instance if you have the capabilities to do so since I cannot guarantee that my server will have a 100% uptime (but I will do my best to keep it online).
I also recommend you to visit this repo from time to time to see if something will change / has already changed with the official instance.
## Self-hosting
_All following instructions are specified for linux, but at least [building](#build-it-from-source) should on every platform too_.
### Docker
**Make sure you have [docker](https://docker.com) installed**.
Clone the repo via `git clone` or download the [zipfile](https://github.com/ByteDream/crunchyroll-go/archive/refs/heads/master.zip) and extract it.
Open a shell, enter the directory and follow the following commands:
```shell
[~/smartrelease]$ docker build -t smartrelease .
[~/smartrelease]$ docker run -p 8080:8080 smartrelease
```
### Binary
Download the latest linux binary from [here](https://smartrelease.bytedream.org/github/ByteDream/smartrelease/smartrelease-v{major}.{minor}.{patch}_linux) (built with musl, so should work on libc and musl systems).
Now simply execute binary and the server is up and running:
```shell
[~]$ ./smartrelease-v<version>_linux
```
### Build it from source
**Make sure you have the latest stable version of [rust](https://www.rust-lang.org/) installed**.
Clone the repo via `git clone` or download the [zipfile](https://github.com/ByteDream/crunchyroll-go/archive/refs/heads/master.zip) and extract it.
Open a shell, enter the directory and follow the following commands:
```shell
[~/smartrelease]$ cargo build --release
[~/smartrelease]$ ./target/release/smartrelease
```
## Configuration
Every configuration can be made with environment variables or via an `.env` file. See [.example.env](.example.env) for an example configuration.
### `HOST`
The host address. I don't really know a case where this has to be changed but to have the choice is always better.
Default is `0.0.0.0`.
### `PORT`
The port to serve the server on.
Default is `8080`.
### `ENABLE_REGEX`
Enable or disable regex support in the pattern.
Default is `false`.
### `MAX_PATTER_LEN`
Limits the maximal length the pattern can have.
Default is `70`.
## Warnings
It is recommended to limit the pattern length with [`MAX_PATTER_LEN`](#max_patter_len) if [`ENABLE_REGEX`](#enable_regex) is enabled since a too long pattern which is too complex could lead to an, wanted or unwanted, [ReDoS](https://en.wikipedia.org/wiki/ReDoS) attack.
If you [host it yourself](#self-hosting) it is highly recommended taking one of the "big" dns servers like `1.1.1.1` or `8.8.8.8` as your dns resolver.
The [actix-web](https://actix.rs/) library which handles all the network stuff sometimes takes up too much time to resolve a dns address when the dns server is, for example, your local router.
And most of the time when this happens a 504 timeout error is thrown and the api is practically unusable.
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for more details.

289
src/main.rs Normal file
View File

@ -0,0 +1,289 @@
use std::{env, io};
use std::collections::HashMap;
use std::io::Write;
use std::str::FromStr;
use std::time::Duration;
use actix_web::{App, dev, Error, get, http, HttpRequest, HttpResponse, HttpServer, Result, web};
use actix_web::client::{Client, Connector};
use actix_web::error::{ErrorInternalServerError, ErrorNotFound, ErrorUriTooLong};
use actix_web::middleware::errhandlers::{ErrorHandlerResponse, ErrorHandlers};
use dotenv::dotenv;
use lazy_static::lazy_static;
use log::info;
use log::LevelFilter::Info;
use regex::{escape, Regex};
use serde::Deserialize;
lazy_static! {
pub static ref ENABLE_REGEX: bool = env_lookup::<bool>("ENABLE_REGEX", false).unwrap();
pub static ref MAX_PATTERN_LEN: i32 = env_lookup::<i32>("MAX_PATTER_LEN", 70).unwrap();
pub static ref TAG_PATTERN: Regex = Regex::new(r"(?P<major>\d+)([.-_](?P<minor>\d+)([.-_](?P<patch>\d+))?([.-_]?(?P<pre>[\w\d]+))?)?").unwrap();
pub static ref REPLACE_PATTERN: Regex = Regex::new(r"\{\w*?}").unwrap();
pub static ref USER_AGENT: String = format!("smartrelease/{}", env_lookup::<String>("CARGO_PKG_VERSION", "".to_string()).unwrap());
}
#[derive(Deserialize)]
struct Query {
/// Normally unmatched wildcards are remaining unedited in the pattern
/// but with this option enabled, these unmatched wildcard are cleared cut out
clear_unknown: Option<bool>,
/// Reverses the asset order provided by the api
reverse: Option<bool>,
// The following fields are alternatives when a wildcard could not be matched
major: Option<String>,
minor: Option<String>,
patch: Option<String>,
pre: Option<String>,
tag: Option<String>
}
#[derive(Deserialize)]
struct Assets {
name: String,
browser_download_url: String
}
#[derive(Deserialize)]
struct GitHub {
tag_name: String,
assets: Vec<Assets>
}
#[get("/github/{user}/{repo}/{pattern}")]
async fn github(
web::Path((user, repo, pattern)): web::Path<(String, String, String)>,
query: web::Query<Query>
) -> Result<HttpResponse> {
if let Some(err) = pre_check(&pattern) {
return Err(err)
}
let mut res = client()
.get(format!("https://api.github.com/repos/{}/{}/releases/latest", user, repo))
.header("Accept", "application/vnd.github.v3+json")
.header("User-Agent", USER_AGENT.as_str())
.send()
.await?;
let mut github = res.json::<GitHub>().await?;
process(&pattern, &mut github.assets, query.into_inner(), &github.tag_name)
}
#[derive(Deserialize)]
struct Gitea {
tag_name: String,
assets: Vec<Assets>
}
#[get("/gitea/{user}/{repo}/{pattern}")]
async fn gitea(
web::Path((user, repo, pattern)): web::Path<(String, String, String)>,
query: web::Query<Query>
) -> Result<HttpResponse> {
let mut res = client()
.get(format!("https://gitea.com/api/v1/repos/{}/{}/releases?limit=1", user, repo))
.header(http::header::CONTENT_TYPE, "application/json")
.header(http::header::USER_AGENT, USER_AGENT.as_str())
.send()
.await?;
let mut gitea = res.json::<[Gitea; 1]>().await?;
return process(&pattern, &mut gitea[0].assets, query.into_inner(), &gitea[0].tag_name)
}
fn redirect_error<B>(res: dev::ServiceResponse<B>) -> Result<ErrorHandlerResponse<B>> {
if res.request().uri().path() == "/favicon" {
return Ok(ErrorHandlerResponse::Response(res))
}
let split_path: Vec<&str> = res.request().uri().path().split("/").collect();
info!("{} {}: got {} ({})",
ip(res.request()),
res.request().path(),
res.status().as_u16(),
res.response().error().map_or_else(|| String::new(), |v| v.to_string()));
if split_path.len() >= 4 {
let location = match *split_path.get(1).unwrap() {
"github" => format!("https://github.com/{}/{}/releases/latest",
*split_path.get(2).unwrap(),
*split_path.get(3).unwrap()),
"gitea" => format!("https://gitea.com/{}/{}/releases",
*split_path.get(2).unwrap(),
*split_path.get(3).unwrap()),
_ => "".to_string()
};
if location != "" {
return Ok(ErrorHandlerResponse::Response(
res.into_response(
HttpResponse::Found()
.header(http::header::LOCATION, location)
.finish()
.into_body()
)
))
}
}
Ok(ErrorHandlerResponse::Response(res))
}
fn client() -> Client {
return Client::builder()
.timeout(Duration::from_secs(5))
.connector(
Connector::new()
.timeout(Duration::from_secs(3))
.finish()
)
.finish();
}
fn env_lookup<F: FromStr>(name: &str, default: F) -> std::result::Result<F, F::Err> {
if let Ok(envvar) = env::var(name) {
return envvar.parse::<F>();
}
Ok(default)
}
fn ip(request: &HttpRequest) -> String {
request
.connection_info().realip_remote_addr().unwrap()
.rsplit_once(":").unwrap().0.to_string()
}
fn pre_check(pattern: &String) -> Option<Error> {
// if MAX_PATTERN_LEN is -1 or below the len checking is disabled
if *MAX_PATTERN_LEN > -1 && REPLACE_PATTERN.replace_all(pattern.as_str(), "").len() > *MAX_PATTERN_LEN as usize {
return Some(ErrorUriTooLong(format!("Pattern / last url path must not exceed {} characters", *MAX_PATTERN_LEN)))
}
None
}
fn process(pattern: &String, assets: &mut Vec<Assets>, query: Query, tag_name: &String) -> Result<HttpResponse> {
let re: Regex;
let mut replaced = replace(
pattern.to_string(),
tag_name.to_string(),
[
("major", query.major),
("minor", query.minor),
("patch", query.patch),
("pre", query.pre),
("tag", query.tag)
].iter().cloned().collect(),
query.clear_unknown.unwrap_or(true)
);
if !*ENABLE_REGEX {
replaced = escape(replaced.as_str())
}
match Regex::new(replaced.as_str()) {
Ok(r) => re = r,
Err(e) => {
return Err(ErrorInternalServerError(e));
}
}
if query.reverse.unwrap_or(false) {
assets.reverse()
}
for asset in assets {
if re.is_match(asset.name.as_str()) {
return Ok(HttpResponse::Found().set_header(http::header::LOCATION, format!("{}", asset.browser_download_url)).finish())
}
}
Err(ErrorNotFound("No matching asset was found"))
}
fn replace(pattern: String, tag: String, alternatives: HashMap<&str, Option<String>>, clear_unknown: bool) -> String {
let mut result = pattern;
if let Some(regex_match) = TAG_PATTERN.captures(tag.as_str()) {
for name in ["major", "minor", "patch", "pre"] {
if let Some(named) = regex_match.name(name) {
result = result.replace(format!("{{{}}}", name).as_str(), named.as_str());
}
}
}
result = result.replace("{tag}", tag.as_str());
for alternative in alternatives {
if let Some(value) = alternative.1 {
result = result.replace(format!("{{{}}}", alternative.0).as_str(), value.as_str());
}
}
if clear_unknown {
result = REPLACE_PATTERN.replace_all(result.as_str(), "").to_string()
}
result
}
#[actix_web::main]
async fn main() -> io::Result<()> {
dotenv().ok();
if env::var("RUST_LOG").is_err() {
env::set_var("RUST_LOG", "actix_server=warn")
}
env_logger::builder()
.format(|buf, record| {
writeln!(
buf,
"[{}] - {}: {}",
buf.timestamp(),
record.level(),
buf.style().value(record.args())
)
})
.filter_level(Info)
.init();
let host = env_lookup::<String>("HOST", "0.0.0.0".to_string()).unwrap();
let port = env_lookup::<i16>("PORT", 8080).unwrap();
let server = HttpServer::new(|| {
App::new()
.service(github)
.service(gitea)
.service(
web::resource("/").route(web::get().to(|| async {
HttpResponse::Found()
.header(http::header::LOCATION, "https://github.com/ByteDream/smartrelease")
.finish()
})
))
.wrap(
ErrorHandlers::new()
.handler(http::StatusCode::BAD_REQUEST, redirect_error)
.handler(http::StatusCode::NOT_FOUND, redirect_error)
.handler(http::StatusCode::GATEWAY_TIMEOUT, redirect_error)
)
})
.bind(format!("{}:{}", host, port))?
.run();
info!(
"Started server on {}:{} with regex {} and a max pattern len of {}",
host,
port,
if *ENABLE_REGEX { "enabled" } else { "disabled" },
*MAX_PATTERN_LEN
);
server.await
}