> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/hub-python-library/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/hub-python-library/reference/managing-local-and-online-repositories.md).

# Managing local and online repositories

## Managing local and online repositories

The `Repository` class is a helper class that wraps `git` and `git-lfs` commands. It provides tooling adapted for managing repositories which can be very large.

It is the recommended tool as soon as any `git` operation is involved, or when collaboration will be a point of focus with the repository itself.

### The Repository class

#### class boincai\_hub.Repository

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L424)

( local\_dir: typing.Union\[str, pathlib.Path]clone\_from: typing.Optional\[str] = Nonerepo\_type: typing.Optional\[str] = Nonetoken: typing.Union\[bool, str] = Truegit\_user: typing.Optional\[str] = Nonegit\_email: typing.Optional\[str] = Nonerevision: typing.Optional\[str] = Noneskip\_lfs\_files: bool = Falseclient: typing.Optional\[boincai\_hub.hf\_api.HfApi] = None )

Helper class to wrap the git and git-lfs commands.

The aim is to facilitate interacting with boincai.com hosted model or dataset repos, though not a lot here (if any) is actually specific to boincai.com.

**\_\_init\_\_**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L435)

( local\_dir: typing.Union\[str, pathlib.Path]clone\_from: typing.Optional\[str] = Nonerepo\_type: typing.Optional\[str] = Nonetoken: typing.Union\[bool, str] = Truegit\_user: typing.Optional\[str] = Nonegit\_email: typing.Optional\[str] = Nonerevision: typing.Optional\[str] = Noneskip\_lfs\_files: bool = Falseclient: typing.Optional\[boincai\_hub.hf\_api.HfApi] = None )

Parameters

* **local\_dir** (`str` or `Path`) — path (e.g. `'my_trained_model/'`) to the local directory, where the `Repository` will be initialized.
* **clone\_from** (`str`, *optional*) — Either a repository url or `repo_id`. Example:
  * `"https://boincai.com/philschmid/playground-tests"`
  * `"philschmid/playground-tests"`
* **repo\_type** (`str`, *optional*) — To set when cloning a repo from a repo\_id. Default is model.
* **token** (`bool` or `str`, *optional*) — A valid authentication token (see [https://boincai.com/settings/token](https://huggingface.co/settings/token)). If `None` or `True` and machine is logged in (through `boincai-cli login` or [login()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/login#huggingface_hub.login)), token will be retrieved from the cache. If `False`, token is not sent in the request header.
* **git\_user** (`str`, *optional*) — will override the `git config user.name` for committing and pushing files to the hub.
* **git\_email** (`str`, *optional*) — will override the `git config user.email` for committing and pushing files to the hub.
* **revision** (`str`, *optional*) — Revision to checkout after initializing the repository. If the revision doesn’t exist, a branch will be created with that revision name from the default branch’s current HEAD.
* **skip\_lfs\_files** (`bool`, *optional*, defaults to `False`) — whether to skip git-LFS files or not.
* **client** (`HfApi`, *optional*) — Instance of [HfApi](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/hf_api#huggingface_hub.HfApi) to use when calling the HF Hub API. A new instance will be created if this is left to `None`.

Raises

*
* * — [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError) if the remote repository set in `clone_from` does not exist.

Instantiate a local clone of a git repo.

If `clone_from` is set, the repo will be cloned from an existing remote repository. If the remote repo does not exist, a `EnvironmentError` exception will be thrown. Please create the remote repo first using [create\_repo()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/hf_api#huggingface_hub.HfApi.create_repo).

`Repository` uses the local git credentials by default. If explicitly set, the `token` or the `git_user`/`git_email` pair will be used instead.

**current\_branch**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L544)

( ) → `str`

Returns

`str`

Current checked out branch.

Returns the current checked out branch.

**add\_tag**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1228)

( tag\_name: strmessage: typing.Optional\[str] = Noneremote: typing.Optional\[str] = None )

Parameters

* **tag\_name** (`str`) — The name of the tag to be added.
* **message** (`str`, *optional*) — The message that accompanies the tag. The tag will turn into an annotated tag if a message is passed.
* **remote** (`str`, *optional*) — The remote on which to add the tag.

Add a tag at the current head and push it

If remote is None, will just be updated locally

If no message is provided, the tag will be lightweight. if a message is provided, the tag will be annotated.

**auto\_track\_binary\_files**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L870)

( pattern: str = '.' ) → `List[str]`

Parameters

* **pattern** (`str`, *optional*, defaults to ”.“) — The pattern with which to track files that are binary.

Returns

`List[str]`

List of filenames that are now tracked due to being binary files

Automatically track binary files with git-lfs.

**auto\_track\_large\_files**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L913)

( pattern: str = '.' ) → `List[str]`

Parameters

* **pattern** (`str`, *optional*, defaults to ”.“) — The pattern with which to track files that are above 10MBs.

Returns

`List[str]`

List of filenames that are now tracked due to their size.

Automatically track large files (files that weigh more than 10MBs) with git-lfs.

**check\_git\_versions**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L559)

( )

Raises

*
* * — [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError) if `git` or `git-lfs` are not installed.

Checks that `git` and `git-lfs` can be run.

**clone\_from**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L582)

( repo\_url: strtoken: typing.Union\[bool, str, NoneType] = None )

Parameters

* **repo\_url** (`str`) — The URL from which to clone the repository
* **token** (`Union[str, bool]`, *optional*) — Whether to use the authentication token. It can be:
  * a string which is the token itself
  * `False`, which would not use the authentication token
  * `True`, which would fetch the authentication token from the local folder and use it (you should be logged in for this to work).
  * `None`, which would retrieve the value of `self.boincai_token`.

Clone from a remote. If the folder already exists, will try to clone the repository within it.

If this folder is a git repository with linked history, will try to update the repository.

Raises the following error:

* [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) if an organization token (starts with “api\_org”) is passed. Use must use your own personal access token (see <https://hf.co/settings/tokens>).
* [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError) if you are trying to clone the repository in a non-empty folder, or if the `git` operations raise errors.

**commit**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1312)

( commit\_message: strbranch: typing.Optional\[str] = Nonetrack\_large\_files: bool = Trueblocking: bool = Trueauto\_lfs\_prune: bool = False )

Parameters

* **commit\_message** (`str`) — Message to use for the commit.
* **branch** (`str`, *optional*) — The branch on which the commit will appear. This branch will be checked-out before any operation.
* **track\_large\_files** (`bool`, *optional*, defaults to `True`) — Whether to automatically track large files or not. Will do so by default.
* **blocking** (`bool`, *optional*, defaults to `True`) — Whether the function should return only when the `git push` has finished.
* **auto\_lfs\_prune** (`bool`, defaults to `True`) — Whether to automatically prune files once they have been pushed to the remote.

Context manager utility to handle committing to a repository. This automatically tracks large files (>10Mb) with git-lfs. Set the `track_large_files` argument to `False` if you wish to ignore that behavior.

Examples:

Copied

```
>>> with Repository(
...     "text-files",
...     clone_from="<user>/text-files",
...     token=True,
>>> ).commit("My first file :)"):
...     with open("file.txt", "w+") as f:
...         f.write(json.dumps({"hey": 8}))

>>> import torch

>>> model = torch.nn.Transformer()
>>> with Repository(
...     "torch-model",
...     clone_from="<user>/torch-model",
...     token=True,
>>> ).commit("My cool model :)"):
...     torch.save(model.state_dict(), "model.pt")
```

**delete\_tag**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1191)

( tag\_name: strremote: typing.Optional\[str] = None ) → `bool`

Parameters

* **tag\_name** (`str`) — The tag name to delete.
* **remote** (`str`, *optional*) — The remote on which to delete the tag.

Returns

`bool`

`True` if deleted, `False` if the tag didn’t exist. If remote is not passed, will just be updated locally

Delete a tag, both local and remote, if it exists

**git\_add**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L988)

( pattern: str = '.'auto\_lfs\_track: bool = False )

Parameters

* **pattern** (`str`, *optional*, defaults to ”.“) — The pattern with which to add files to staging.
* **auto\_lfs\_track** (`bool`, *optional*, defaults to `False`) — Whether to automatically track large and binary files with git-lfs. Any file over 10MB in size, or in binary format, will be automatically tracked.

git add

Setting the `auto_lfs_track` parameter to `True` will automatically track files that are larger than 10MB with `git-lfs`.

**git\_checkout**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1129)

( revision: strcreate\_branch\_ok: bool = False )

Parameters

* **revision** (`str`) — The revision to checkout.
* **create\_branch\_ok** (`str`, *optional*, defaults to `False`) — Whether creating a branch named with the `revision` passed at the current checked-out reference if `revision` isn’t an existing revision is allowed.

git checkout a given revision

Specifying `create_branch_ok` to `True` will create the branch to the given revision if that revision doesn’t exist.

**git\_commit**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1022)

( commit\_message: str = 'commit files to HF hub' )

Parameters

* **commit\_message** (`str`, *optional*, defaults to “commit files to HF hub”) — The message attributed to the commit.

git commit

**git\_config\_username\_and\_email**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L710)

( git\_user: typing.Optional\[str] = Nonegit\_email: typing.Optional\[str] = None )

Parameters

* **git\_user** (`str`, *optional*) — The username to register through `git`.
* **git\_email** (`str`, *optional*) — The email to register through `git`.

Sets git username and email (only in the current repo).

**git\_credential\_helper\_store**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L729)

( )

Sets the git credential helper to `store`

**git\_head\_commit\_url**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L766)

( ) → `str`

Returns

`str`

The URL to the current checked-out commit.

Get URL to last commit on HEAD. We assume it’s been pushed, and the url scheme is the same one as for GitHub or BOINC AI.

**git\_head\_hash**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L738)

( ) → `str`

Returns

`str`

The current checked out commit SHA.

Get commit sha on top of HEAD.

**git\_pull**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L964)

( rebase: bool = Falselfs: bool = False )

Parameters

* **rebase** (`bool`, *optional*, defaults to `False`) — Whether to rebase the current branch on top of the upstream branch after fetching.
* **lfs** (`bool`, *optional*, defaults to `False`) — Whether to fetch the LFS files too. This option only changes the behavior when a repository was cloned without fetching the LFS files; calling `repo.git_pull(lfs=True)` will then fetch the LFS file from the remote repository.

git pull

**git\_push**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1039)

( upstream: typing.Optional\[str] = Noneblocking: bool = Trueauto\_lfs\_prune: bool = False )

Parameters

* **upstream** (`str`, *optional*) — Upstream to which this should push. If not specified, will push to the lastly defined upstream or to the default one (`origin main`).
* **blocking** (`bool`, *optional*, defaults to `True`) — Whether the function should return only when the push has finished. Setting this to `False` will return an `CommandInProgress` object which has an `is_done` property. This property will be set to `True` when the push is finished.
* **auto\_lfs\_prune** (`bool`, *optional*, defaults to `False`) — Whether to automatically prune files once they have been pushed to the remote.

git push

If used without setting `blocking`, will return url to commit on remote repo. If used with `blocking=True`, will return a tuple containing the url to commit and the command object to follow for information about the process.

**git\_remote\_url**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L751)

( ) → `str`

Returns

`str`

The URL of the `origin` remote.

Get URL to origin remote.

**is\_repo\_clean**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1262)

( ) → `bool`

Returns

`bool`

`True` if the git status is clean, `False` otherwise.

Return whether or not the git status is clean or not

**lfs\_enable\_largefiles**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L856)

( )

HF-specific. This enables upload support of files >5GB.

**lfs\_prune**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L946)

( recent = False )

Parameters

* **recent** (`bool`, *optional*, defaults to `False`) — Whether to prune files even if they were referenced by recent commits. See the following [link](https://github.com/git-lfs/git-lfs/blob/f3d43f0428a84fc4f1e5405b76b5a73ec2437e65/docs/man/git-lfs-prune.1.ronn#recent-files) for more information.

git lfs prune

**lfs\_track**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L815)

( patterns: typing.Union\[str, typing.List\[str]]filename: bool = False )

Parameters

* **patterns** (`Union[str, List[str]]`) — The pattern, or list of patterns, to track with git-lfs.
* **filename** (`bool`, *optional*, defaults to `False`) — Whether to use the patterns as literal filenames.

Tell git-lfs to track files according to a pattern.

Setting the `filename` argument to `True` will treat the arguments as literal filenames, not as patterns. Any special glob characters in the filename will be escaped when writing to the `.gitattributes` file.

**lfs\_untrack**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L840)

( patterns: typing.Union\[str, typing.List\[str]] )

Parameters

* **patterns** (`Union[str, List[str]]`) — The pattern, or list of patterns, to untrack with git-lfs.

Tell git-lfs to untrack those files.

**list\_deleted\_files**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L780)

( ) → `List[str]`

Returns

`List[str]`

A list of files that have been deleted in the working directory or index.

Returns a list of the files that are deleted in the working directory or index.

**push\_to\_hub**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1276)

( commit\_message: str = 'commit files to HF hub'blocking: bool = Trueclean\_ok: bool = Trueauto\_lfs\_prune: bool = False )

Parameters

* **commit\_message** (`str`) — Message to use for the commit.
* **blocking** (`bool`, *optional*, defaults to `True`) — Whether the function should return only when the `git push` has finished.
* **clean\_ok** (`bool`, *optional*, defaults to `True`) — If True, this function will return None if the repo is untouched. Default behavior is to fail because the git command fails.
* **auto\_lfs\_prune** (`bool`, *optional*, defaults to `False`) — Whether to automatically prune files once they have been pushed to the remote.

Helper to add, commit, and push files to remote repository on the BOINC AI Hub. Will automatically track large files (>10MB).

**tag\_exists**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1161)

( tag\_name: strremote: typing.Optional\[str] = None ) → `bool`

Parameters

* **tag\_name** (`str`) — The name of the tag to check.
* **remote** (`str`, *optional*) — Whether to check if the tag exists on a remote. This parameter should be the identifier of the remote.

Returns

`bool`

Whether the tag exists.

Check if a tag exists or not.

**wait\_for\_commands**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1442)

( )

Blocking method: blocks all subsequent execution until all commands have been processed.

### Helper methods

**boincai\_hub.repository.is\_git\_repo**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L113)

( folder: typing.Union\[str, pathlib.Path] ) → `bool`

Parameters

* **folder** (`str`) — The folder in which to run the command.

Returns

`bool`

`True` if the repository is part of a repository, `False` otherwise.

Check if the folder is the root or part of a git repository

**boincai\_hub.repository.is\_local\_clone**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L130)

( folder: typing.Union\[str, pathlib.Path]remote\_url: str ) → `bool`

Parameters

* **folder** (`str` or `Path`) — The folder in which to run the command.
* **remote\_url** (`str`) — The url of a git repository.

Returns

`bool`

`True` if the repository is a local clone of the remote repository specified, `False` otherwise.

Check if the folder is a local clone of the remote\_url

**boincai\_hub.repository.is\_tracked\_with\_lfs**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L155)

( filename: typing.Union\[str, pathlib.Path] ) → `bool`

Parameters

* **filename** (`str` or `Path`) — The filename to check.

Returns

`bool`

`True` if the file passed is tracked with git-lfs, `False` otherwise.

Check if the file passed is tracked with git-lfs.

**boincai\_hub.repository.is\_git\_ignored**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L192)

( filename: typing.Union\[str, pathlib.Path] ) → `bool`

Parameters

* **filename** (`str` or `Path`) — The filename to check.

Returns

`bool`

`True` if the file passed is ignored by `git`, `False` otherwise.

Check if file is git-ignored. Supports nested .gitignore files.

**boincai\_hub.repository.files\_to\_be\_staged**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L240)

( pattern: str = '.'folder: typing.Union\[str, pathlib.Path, NoneType] = None ) → `List[str]`

Parameters

* **pattern** (`str` or `Path`) — The pattern of filenames to check. Put `.` to get all files.
* **folder** (`str` or `Path`) — The folder in which to run the command.

Returns

`List[str]`

List of files that are to be staged.

Returns a list of filenames that are to be staged.

**boincai\_hub.repository.is\_tracked\_upstream**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L265)

( folder: typing.Union\[str, pathlib.Path] ) → `bool`

Parameters

* **folder** (`str` or `Path`) — The folder in which to run the command.

Returns

`bool`

`True` if the current checked-out branch is tracked upstream, `False` otherwise.

Check if the current checked-out branch is tracked upstream.

**boincai\_hub.repository.commits\_to\_push**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L287)

( folder: typing.Union\[str, pathlib.Path]upstream: typing.Optional\[str] = None ) → `int`

Parameters

* **folder** (`str` or `Path`) — The folder in which to run the command.
* **upstream** (`str`, *optional*) —

Returns

`int`

Number of commits that would be pushed upstream were a `git push` to proceed.

Check the number of commits that would be pushed upstream

The name of the upstream repository with which the comparison should be made.

### Following asynchronous commands

The `Repository` utility offers several methods which can be launched asynchronously:

* `git_push`
* `git_pull`
* `push_to_hub`
* The `commit` context manager

See below for utilities to manage such asynchronous methods.

#### class boincai\_hub.Repository

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L424)

( local\_dir: typing.Union\[str, pathlib.Path]clone\_from: typing.Optional\[str] = Nonerepo\_type: typing.Optional\[str] = Nonetoken: typing.Union\[bool, str] = Truegit\_user: typing.Optional\[str] = Nonegit\_email: typing.Optional\[str] = Nonerevision: typing.Optional\[str] = Noneskip\_lfs\_files: bool = Falseclient: typing.Optional\[boincai\_hub.hf\_api.HfApi] = None )

Helper class to wrap the git and git-lfs commands.

The aim is to facilitate interacting with boincai.com hosted model or dataset repos, though not a lot here (if any) is actually specific to boincai.com.

**commands\_failed**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1428)

( )

Returns the asynchronous commands that failed.

**commands\_in\_progress**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1435)

( )

Returns the asynchronous commands that are currently in progress.

**wait\_for\_commands**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L1442)

( )

Blocking method: blocks all subsequent execution until all commands have been processed.

#### class boincai\_hub.repository.CommandInProgress

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/repository.py#L30)

( title: stris\_done\_method: typing.Callablestatus\_method: typing.Callableprocess: Popenpost\_method: typing.Optional\[typing.Callable] = None )

Utility to follow commands launched asynchronously.